When the Measure Becomes the Target

I once sat in on a quarterly review where an engineering team celebrated crossing 90% test coverage — up from 61% in a single quarter, a genuinely heroic-sounding climb. There was applause. There was, I'm told, cake. Three months later the same team had its worst quarter of production incidents in two years. When a curious staff engineer finally audited the test suite, she found the mechanism: hundreds of tests that invoked functions and asserted nothing. Not sloppy tests — assertion-free tests, some visibly generated in bulk, executing code purely so the coverage tool would watch it run. The suite had become a machine for turning lines green.

Nobody on that team was dishonest. Every individual decision was locally reasonable: the target was 90%, the deadline was real, and an executed line counts the same whether or not anyone checks what it did. The team did exactly what the metric asked. That's the whole problem — and it has a name.

Goodhart, Strathern, Campbell: a short genealogy

The economist Charles Goodhart observed in the 1970s — originally about monetary policy — that statistical regularities tend to collapse once pressure is placed upon them for control purposes. The version everyone quotes is actually anthropologist Marilyn Strathern's later paraphrase: "When a measure becomes a target, it ceases to be a good measure."

The social scientist Donald Campbell got there independently, with a darker emphasis. Campbell's law holds that the more any quantitative indicator is used for decision-making, the more subject it becomes to corruption pressures — and the more apt it is to distort and corrupt the very processes it was meant to monitor. Note the second clause: the indicator doesn't just become inaccurate. It reaches back and deforms the underlying work.

Why is this so reliable it earns the word "law"? Because every metric is a proxy. We can't measure "software quality" or "engineering productivity" directly — they're multidimensional and partly unobservable — so we measure a shadow: coverage, story points, PR counts, lead time. A proxy correlates with the real thing under the behaviors that existed when you chose it. Attach incentives, and behavior migrates toward whatever moves the proxy at least cost. The cheapest paths to a better number are almost never the paths through better reality — reality is expensive; the number is negotiable. The correlation you were counting on is destroyed by the very act of counting on it.

How metrics reshape behavior, quietly

The word I want to stress is quietly. Goodhart failures rarely look like gaming, least of all to the people doing them. They look like prioritization.

An engineer with a coverage target doesn't think "I'll write fraudulent tests." They think "I'll test the easy modules first" — the pure functions, the getters — while the gnarly concurrent core, where the bugs actually live, stays conveniently expensive. A team measured on velocity doesn't conspire to inflate estimates; estimates simply... drift, three points becoming five, until the chart pleases. Measure closed tickets and tickets fission into confetti. Measure PR count and — a fresh classic — an AI assistant will happily help anyone manufacture a blizzard of small, plausible pull requests whose review cost lands on someone else's ledger.

Each actor optimizes in good faith. The distortion only exists in aggregate, which is why nobody feels responsible for it and why dashboards stay green while the thing they allegedly measure decays. The metric hasn't just stopped tracking quality; per Campbell, it has redirected real effort away from quality toward number-tending. You paid salary for the gaming.

And there's a second-order cost that I think is underrated: a gamed metric is worse than no metric, because it manufactures false confidence. The team with no coverage number knows it's flying blind and behaves with appropriate fear. The team with a gamed 90% has a certificate saying it's safe — my cake-eating friends discovered exactly what that certificate was worth.

Coverage, velocity, and other well-behaved liars

Let me be fair to the metrics themselves, because none of them started out as liars.

Coverage is a superb diagnostic. As a question — "which parts of the system do our tests never even execute?" — it finds risk with precision; a module at 15% is telling you something true. As a target, it measures execution, not verification, and execution can be manufactured. The tragedy is asymmetric: low coverage reliably indicates missing tests, but high coverage indicates nothing about whether the tests can fail. (If you want a target-grade signal, mutation testing — checking whether seeded bugs actually make tests fail — is far harder to game, precisely because it measures verification.)

Velocity was invented as a forecasting input — a team-local calibration constant for planning. Promoted to a performance target, or worse, compared across teams, it corrupts in weeks, because the unit is self-reported and elastic. You can't inflate reality, but you can always inflate a point.

Lead time, deployment frequency, and their DORA cousins are better than most, because they're harder to fake without approximating the real behavior. But even here, the researchers behind them have been explicit that they work as capability signals for learning, not as weapons for ranking — the moment a number decides bonuses, Campbell's corruption pressure applies to it like gravity.

The pattern: every one of these is honest as a thermometer and corruptible as a thermostat. Reading the temperature is science. Wiring the temperature to consequences invites everyone to warm the sensor with a match.

Measuring without corrupting

I am not going to conclude "don't measure" — unmeasured engineering has failure modes of its own, and vibes are just metrics with worse error bars. The craft is measuring in ways that resist Goodhart. Four practices:

Keep proxies as questions, not verdicts. "Coverage dropped 8% in the payments module — what happened?" is a healthy use. "All modules above 85% by Q3" is an invitation to assertion-free theater. The moment a number closes a conversation instead of opening one, it's become a target.

Pair every metric with a counter-metric. Goodhart exploits the unmeasured dimension, so measure the dimension gaming would sacrifice: velocity paired with escaped defects; PR count paired with revert rate; coverage paired with mutation score or incident data. Gaming one number while its counterweight watches is much more expensive.

Prefer outcomes to activity. Incidents, user-visible reliability, time-to-restore sit closer to what you actually value than lines, points, or counts. The nearer the metric is to reality, the more that gaming it approximates doing the job — which is the one failure mode you can live with.

Rotate and retire. Correlations die under pressure; treat metrics as having half-lives. When a number has been a target long enough to be gamed, thank it for its service and vary the instrument. Fixed targets breed fixed exploits.

The seminar summary

1. Recite Strathern before every dashboard meeting: when a measure becomes a target, it ceases to be a good measure. It's cheaper to remember than to rediscover.
2. Audit one green number this month. Pick your team's proudest metric and ask what behavior, exactly, has been producing it. Read ten of the tests behind the coverage figure. Bring snacks; sometimes there's archaeology.
3. Ask the two-question test of any proposed target: What's the cheapest way to move this number without improving anything? Who notices if someone does? No good answers, no target.
4. Use numbers to start conversations, judgment to end them. The metric spots the anomaly; humans decide what it means. That division of labor is the whole game.

Measurement is how we see. Goodhart's law is the reminder that the act of watching changes the watched — so build instruments you can trust because everyone knows their limits, not despite them. And maybe hold the cake until the incidents say what the dashboard can't.