ninjahawk/livenerf

I really like this idea. Maybe I'm already doing it 🤔

All part of the plan.

One way to turn this from a weakness into a test: run a second, secret panel from separate, unrelated accounts with irregular timing, and don't publish its contents or results until the series ends. If the public panel holds steady while the secret one drops, that divergence is itself evidence of benchmark-specific routing. It also changes the incentives. Gaming a known benchmark carries a much bigger reputational risk than a quiet nerf, since being caught with a defeat device is worse than being caught with a regression. A holdout makes the cost of being caught real. The repo's procedural generators could supply secret items with known difficulty that don't appear in any public dataset.