The AI Speed You Feel Isn't the Speed You Ship

Start with the study everyone argues about. METR ran a randomized controlled trial in 2025 with 16 experienced open-source developers on 246 real tasks, in repositories they personally maintained, averaging over a million lines of code. Half the tasks allowed AI tools. Half didn't.

The AI-allowed tasks took 19% longer.

That's not the interesting part. The interesting part is that going in, those developers predicted AI would make them 24% faster. Afterward, having lived through the slowdown, they still believed they'd been about 20% faster. METR called it a substantial and persistent gap between perceived and actual performance.

They were wrong about their own work while doing it. That should make anyone nervous.

2026 data points the same direction from a different angle. A Hanover Research study found 86% of organizations reported an increase in senior engineer firefighting, and 74% said at least a quarter of AI-generated code needed significant rework after deployment.

After deployment. Weeks later. Long after the sprint where it felt fast.

No, and anyone selling you that reading is doing the same thing as the vendors, just in the other direction.

Read the METR setup carefully. Experienced developers. Codebases they knew intimately, over a million lines. That's close to the hardest possible case for an AI assistant, because the human already holds more context than the model can be given.

Invert every one of those conditions and you get a first-time founder writing a new product. Small codebase. Unfamiliar framework. No accumulated context to lose. In that setting the tools are extraordinary, and pretending otherwise is silly.

So the useful summary is narrower than either camp wants: AI accelerates writing code you don't know how to write, in systems that don't have much history yet. It decelerates careful work in systems you already understand.

Most startups are in the first category on day one and drift into the second by month eighteen. Nobody notices the drift.

Four places, and none of them show up on a burndown chart.

Reviewing suggestions. You read output you didn't write, which is a different and slower cognitive task than writing it yourself. Reading unfamiliar code is real work.

Re-prompting. The first answer is close but wrong, so you try again. Three attempts feel like thirty seconds each. They aren't.

Debugging plausible code. AI-generated code fails in a specific way: it looks right. Bugs that survive review are the expensive kind, because they've earned your trust before they break.

Context switching. You bounce between holding your own model of the problem and evaluating someone else's. That switch has a real cost and it repeats hundreds of times a day.

Then there's the downstream version. Pull request volume goes up because writing got faster. Review queues grow. QA saturates. Security validation lags. Your team's throughput is capped by the slowest stage, and coding was almost never the slowest stage.

More than the subscription line item suggests.

Seat-based coding tools run roughly $19 to $200 per user per month. That's the number founders budget. Agentic tools that run longer autonomous loops are metered on tokens, and reported spend runs $200 to $2,000 or more per engineer per month depending on usage. Blended, teams are landing around $200 to $600 per engineer monthly.

For a four-person engineering team that's $10,000 to $29,000 a year, which is fine if it works and expensive if it doesn't.

The rework cost is larger and invisible. If a quarter of AI-generated code needs significant post-deployment work, and your senior engineer is the one doing it, you're paying your most expensive person to clean up your cheapest output. That's the actual bill.

Worth putting real numbers against this in whatever you use to model burn. Foundra's financial projection templates have a line for tooling that most first-time founders leave at the sticker price; token spend and senior rework hours belong there too. Even a spreadsheet with two extra rows beats guessing.

Differently depending on what you're building at that moment. Three modes.

Prototype mode. You're testing whether an idea works at all. Use everything. Generate freely, don't review carefully, expect to throw it away. Speed is the only thing that matters and correctness really doesn't. This is where AI is at its best and most founders underuse it here.

Product mode. Real users depend on this. Use AI for the parts you'd otherwise copy from documentation: boilerplate, tests, migrations, glue. Write the core logic yourself. Review everything as if a contractor wrote it, because effectively one did.

Maintenance mode. Something is broken in production. This is the METR condition, and it's where AI most reliably slows you down. Read the code. Think. Use the model as a rubber duck, not a driver.

The mistake isn't picking the wrong tool. It's staying in prototype mode after the product stopped being a prototype.

Velocity is the metric that lies here, because it counts output and the problem is downstream.

Track change failure rate: what percentage of deploys cause an incident or need a rollback. If it climbs after you adopt a new tool, the speed is fake.

Track time from pull request opened to merged. If writing got faster and this got slower, you moved the bottleneck instead of removing it.

Track how much of your senior engineer's week goes to unplanned work. Hanover's 86% firefighting number is the canary. If your most experienced person spends Thursday fixing Tuesday's merge, you're accumulating debt at a rate nobody's writing down.

And track one soft thing: ask the team monthly whether they understand the code they shipped. Not whether it works. Whether they understand it. A team that ships code it can't explain is a team that will be very slow in six months, and no dashboard catches it.

Yes, and this is where founders get it backwards.

The pitch is that AI lets you hire fewer or more junior engineers, because the model handles the hard parts. The data says the opposite. When output volume rises, review capacity becomes the constraint, and review is a senior skill. You need more judgment per line, not less.

So the first engineering hire matters more now, not less. Someone who can look at generated code and say "this works but it'll break when we have 10,000 rows" is worth several people who can produce more code.

The flip side is real too. A small team with good judgment can now cover surface area that used to need eight people. That's the actual gain. It just doesn't arrive by hiring cheaper, it arrives by hiring better and staying small.

One practical note. When you interview, ask candidates to review AI-generated code with a bug in it rather than write code from scratch. It tests the skill you'll actually need.

• METR's randomized trial found experienced developers were 19% slower with AI tools, while believing they were 20% faster.
• 2026 Hanover Research data: 86% of organizations saw more senior engineer firefighting, 74% said a quarter or more of AI code needed significant post-deployment rework.
• The tools help most on unfamiliar work in young codebases and hurt most on careful work in systems you know well.
• Real cost runs $200 to $600 per engineer per month blended, plus senior hours spent on rework that nobody budgets.
• Use three modes: generate freely in prototypes, review hard in product, mostly think in maintenance.
• Measure change failure rate, PR cycle time, and unplanned senior work instead of raw velocity.
• Faster writing raises the value of senior review. Hire for judgment, not volume.

Did the METR study prove AI makes developers slower? It found a 19% slowdown for 16 experienced developers on 246 tasks in large codebases they maintained themselves. That's a specific, difficult setting. It doesn't generalize to a new project or an unfamiliar stack, where the same tools usually help.

Why do developers think AI speeds them up when it doesn't? Because the parts that feel slow, staring at a blank file, looking things up, get faster, while the parts that grow, reviewing and debugging generated code, feel like normal work. The costs are distributed and the benefits are concentrated at a memorable moment.

How much should a startup budget for AI coding tools? Blended reports land around $200 to $600 per engineer per month once you include both seat licenses and token-metered agentic usage. Budget the range, not the sticker price, and revisit after two months of actual invoices.

What is post-deployment rework? Work done on code after it's already live: fixing defects, handling edge cases that weren't considered, refactoring something that doesn't scale. It's expensive because it usually falls to senior engineers and interrupts planned work.

Should early-stage startups still use AI coding tools? Yes, especially before product-market fit, when most code is disposable and speed of learning matters more than durability. The discipline is switching modes once real users depend on the system.

Does AI mean I can hire fewer engineers? Not in the way most founders assume. It lets a small experienced team cover more ground, but it increases the amount of review needed per unit of output, and review is a senior skill. Hire better rather than cheaper.

1. METR: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
2. Larridin: The Hidden Costs of AI Coding Tools Most Engineering Leaders Miss
3. Larridin: Developer Productivity Benchmarks 2026
4. Exceeds: AI Coding Agent Productivity Debates, The 2026 Paradox
5. Keyhole Software: AI Software Development Costs 2026, Enterprise Spending, TCO and ROI Analysis
6. Dark Reading: AI Coding, Do Security Risks Outweigh Productivity Gains?