There is a famous story about British rule in India. Officials in Delhi, alarmed by the number of venomous cobras, offered a bounty for every dead snake brought to the collection office. At first, the policy worked beautifully. Cobras were killed, rewards were collected, and officials watched the numbers improve.
Then, people began breeding cobras!
The bounty had transformed a dangerous animal into a profitable asset. When the government discovered the scheme and canceled the program, breeders released their now-worthless stock. The cobra population ended up higher than before the bounty began.
Economists call this the Cobra Effect: attach a reward to an imperfect measure of success, and people learn to improve the measure without producing the success. The proxy looks great. The underlying reality gets worse. In May 2026, Amazon rediscovered it with artificial intelligence.
Amazon built an internal leaderboard called KiroRank to encourage engineers to use its AI coding tools. The leaderboard rewarded something easy to count: token consumption. Engineers started running pointless AI tasks to climb the rankings. The practice got a name: tokenmaxxing.
On May 29, 2026, Amazon deprecated the leaderboard. They shifted to “normalized deployments”: code that actually ships. [1]
Meta had built its own tracker, Claudeonomics, across 85,000 employees. Over 30 days, employees consumed over 60 trillion tokens. The tracker was reportedly discontinued in April 2026. [2]
In 1975, Charles Goodhart [3] noticed the same pattern in monetary policy. Once a statistical indicator was used for control, the relationship that made it useful collapsed, noted as:
When a measure becomes a target, it ceases to be a good measure.
The mechanism: every measurement is a proxy for something you actually care about. As long as the proxy and the real goal move together, the proxy is useful. The trouble starts when you optimize the proxy.
Researchers at OpenAI demonstrated this in 2023. Under sufficiently strong optimisation, the proxy reward continued to rise while a gold-standard measure of quality peaked and then deteriorated.
Education. Teachers measured by test scores stop teaching the subject and start teaching the test. Wells Fargo. Employees opened 3.5 million fake accounts to hit sales quotas. $3 billion in fines. Click-through rate. Optimize for clicks, and you get rage-bait. The proxy goes up. Trust erodes.
The same problem appears inside language models. In RLHF, a reward model predicts human approval, and the model is optimized to match that prediction. Under strong optimisation, the model becomes better at satisfying the evaluator without becoming better at the task.
One failure mode is sycophancy: accommodating beliefs because agreement is rewarded. Another is subtler: models learn to make incorrect answers more persuasive to evaluators without improving correctness. That is Goodhart’s Law inside a training loop.
*Proxies exist because the true outcome is hard to measure in real time. *The answer is not “stop using proxies.” It is to treat them as hypotheses that need periodic testing and fine-tuning.
1. The Shadow Incentive. What is the cheapest way to move this number without doing the work?
2. The Proxy Gap. Is this metric measuring the outcome, or just the trace of the action?
3. The Dark Matter. What important behaviour is invisible to the metric?
4. The Divergence Check. Does the proxy still predict the outcome?
5. The Adaptation Test. Can the metric survive after everyone learns the scoring rule?
The audit catches the symptom. The next question is how to build metrics that resist gaming in the first place.
In online experimentation, we use an Overall Evaluation Criterion (OEC): a composite metric that combines what you want with penalties for what you want to avoid. Consider an email team that measures revenue but subtracts a penalty for each unsubscribe, weighted by its estimated lifetime cost:
OEC = (Revenue − Unsubscribes × Estimated_Cost) / Users// Cohort analysis: each unsubscribe costs ~$35 in future revenue// A campaign: $10,000 revenue, 300 unsubscribes// OEC = $10,000 − (300 × $35) = −$500// Revenue alone says ship. The OEC says kill it.OEC = w1 * primary_metric + w2 * retention_signal − λ1 * known_sacrifice − λ2 * cost
The penalty term turns the side effect of gaming into a cost that shows up in the score. Why this matters: without penalty terms, even well-intentioned metrics can hide failure:
In one Bing experiment [4], a ranking bug degraded search results. Yet distinct queries rose by more than 10%, and revenue rose by more than 30%, because frustrated users had to search repeatedly.** Two seemingly positive metrics improved not because the product had become better, but because users had to work harder.** A stronger version supported by task-success measures and guardrails would have exposed the deterioration.
The system needs three layers:
1. The OEC decides whether to ship. Balances short-term and long-term.
2. Guardrail metrics must not deteriorate (latency, errors, crashes). If breached, the experiment is aborted regardless of the OEC.
3. Diagnostic metrics explain why the OEC moved. They carry understanding, not incentives.
The OEC itself is still a proxy for long-term value. Its weights can be wrong, its components can miss important harms, and the relationships it depends on can weaken once people learn the scoring rule.
Netflix iterated through four versions over two years before finding one that reliably predicted 90-day retention. A good OEC must be validated against real outcomes and revised when the link weakens.
So, before putting a KPI on a dashboard, a leaderboard, or a training loop, ask yourself:
If someone cared only about making this number rise, what would they do?
Then design around the answer. Put the intended outcome in the decision criterion—price known sacrifices into the score. Protect what must not deteriorate with guardrails. Keep usage metrics diagnostic rather than aspirational.
The goal is not to invent a metric that can never be gamed. It is to build a measurement system that notices when the game has changed.
Because every time an organisation mistakes a proxy for the goal, somewhere a cobra breeder starts drafting a business plan!
The Cobra Effect, Running on GPUs was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.