Engineering Metrics in the Age of AI

June 21, 2026

Sometime in your first year, a board member will ask whether AI made your engineering team more productive. You will have about ninety seconds and one slide.

This article is about what belongs on that slide.

The trap is that you will not lack numbers. You will have too many, most of them flattering, and every one of them answering a question nobody asked.

Why the Easy Answers Fail#

In 1982, Apple asked Bill Atkinson to report how many lines of code he wrote each week. He had just rewritten a core graphics engine to be faster, smaller, and simpler - so he wrote “-2000.” They stopped sending him the form¹.

Engineering has known its output numbers were broken for forty years. We kept using them anyway, for one quiet reason: producing output was expensive.

That cost kept the numbers accidentally honest. Counting pull requests worked the way counting receipts works - forging them took effort. The old rule that a measure stops working once it becomes a target² stayed half-tamed, because gaming your commit count still meant writing the code.

AI removed the effort. An engineer with an agent can multiply their commits, pull requests, and story points in a week without the product improving at all.

This is happening at platform scale, not just inside your company. In June, GitHub shipped a cap on how many pull requests an outside contributor can have open at once, explaining that maintainers “are dealing with an ever-growing volume of pull requests, including repeated low-quality or drive-by contributions that can slow triage and overwhelm review queues.”³

A volume number that platforms now have to cap is not a volume number that can stand for value.

Diagnostic, Not Dead#

Here is where most of the commentary overshoots, and where I overshot in an earlier version of this piece.

The tempting conclusion is that these numbers are now worthless and should be banned. That is wrong, and a board member who has run an organization will spot it immediately. Volume numbers still carry real information. They have simply lost the one job they were never good at.

The distinction that matters is not good metric versus bad metric. It is diagnostic versus performance measure - what a number can tell you when you are investigating, against what it can prove when you are judging.

Every one of those left-hand numbers is useful on a Tuesday when you are trying to work out why a team is stuck.

None of them belongs on the board slide.

The failure mode is not measurement. It is promotion - taking a number that was doing honest diagnostic work and asking it to stand for productivity, which it was never able to do and can now be inflated at will.

That reframing matters practically. “Ban this metric” loses the argument with a CEO who wants a number. “This is a diagnostic, and here is what I use instead when the question is performance” wins it.

What the Honest Numbers Show#

Strip away the keynotes and the measured picture is narrower than either the enthusiasts or the skeptics want.

The industry’s longest-running delivery research has now covered AI adoption in two consecutive annual reports. Between them, the throughput picture improved. The stability picture did not⁴⁵. Inside the codebase, the two health indicators move in opposite directions: duplication up, refactoring down⁶.

And the most useful benchmark I have found puts the median gain in pull-request throughput at 7.8%, with most organizations landing between 5% and 15%⁷. This is vendor research rather than peer review, drawn from more than 400 companies, and it is worth noting that it measures throughput of pull requests - itself a generation-side proxy. Even the friendly number, measured on friendly terms, is single digits.

Genuinely useful. An order of magnitude below what is claimed on stage.

Two of the three claims currently circulating in board decks fail on definition alone. Microsoft and Google have both cited figures around 30% of code written by AI, without a shared definition of what counts⁸ - it is the lines-of-code metric reborn with better marketing. Suggestion acceptance rate measures how agreeable your engineers are. And near-universal AI usage⁵ is table stakes, not a result.

If your dashboard says AI tripled your productivity, your dashboard is measuring generation rather than delivery. Why those two diverge is its own article⁹.

The Paired Scorecard#

The numbers that survive share one property: they describe the system, not a person. No amount of individual volume can fake them. As Dan North puts it, they “measure the engine, not the contribution of individual pistons.”¹⁰

But system metrics alone are still gameable in one direction - you can always buy speed with quality. So the working rule is: never put a speed number on the slide without the counter-metric that catches what it costs.

Three rules of hygiene make it work. Measure teams and services rather than individuals. Watch trends rather than levels - you are looking for a direction, not a grade. And give every number an owner and a threshold, because a metric that cannot trigger a decision is decoration¹².

The Ninety Seconds#

Here is the scorecard doing its job in an actual sentence:

“Delivery lead time improved 14% this half, and our change-failure rate stayed flat. We read that as a real gain rather than volume moving downstream - if the speed were coming out of quality, failures would have risen with it. Rework is up slightly, so we are cutting batch size next quarter. We are not reporting an AI productivity number, because the ones available measure how much the tools are used rather than what shipped.”

That answer takes twenty seconds, survives a follow-up question, and does something the “AI made us 40% faster” answer cannot: it tells the board what you will do next.

The last sentence is the one that takes nerve. Say it anyway. Declining to report a bad number is a stronger position than reporting one you will have to walk back when the quarter does not match it.

Where I Would Push Back on Myself#

The strongest objection to everything above is that “never measure individuals” is too convenient.

Managers do have to evaluate individuals. Promotions, performance reviews, and the occasional decision to let someone go are real and unavoidable. Refusing to measure people does not make those judgments disappear. It just moves them somewhere unaccountable - into visibility, likeability, and how recently someone spoke in a meeting. That is not obviously fairer than a flawed number, and for people who are quiet or remote or new, it is usually worse.

So I would state the rule more narrowly than I used to.

Individual assessment is legitimate. Individual metrics are the problem, and specifically metrics derived from output volume, because those are exactly the ones AI can now inflate on demand. What survives at the individual level is evidence a person’s peers would recognize: what they own, what they unblocked, whether their reviews improve the work, whether their judgment is trusted on the thing they are responsible for.

That is harder than a dashboard. It is also the part of management that was never going to be automated.

Closing#

AI did not break engineering metrics. It broke the illusion that the broken ones were working - teams already measuring outcomes barely had to change anything, and everyone else lost their alibi.

Atkinson’s weekly form had it backwards in 1982 and it is still backwards now. Some of the best engineering work of the next few years will look like negative two thousand lines, and your scorecard should be able to tell.

References#