Google’s 2025 DORA report surveyed close to 5,000 technology professionals. Ninety percent now use AI at work. More than 80 percent believe it has made them more productive. For the first time, AI adoption showed a positive relationship with software delivery throughput, reversing the previous year’s finding.
The same report found that AI adoption still has a negative relationship with delivery stability. More change failures, more rework, longer recovery.
DORA’s own explanation of that tension is the interesting part. Around 30 percent of developers report little or no trust in AI-generated code. Because current tools cannot signal their own uncertainty and produce wrong answers with the same confidence as right ones, engineers audit everything. Time saved during generation moves into verification overhead.
That is the pattern worth taking seriously. Generation got cheap. The checking did not, and it is now where the work sits.
The cost of checking is invisible to the people paying it
METR ran a randomized controlled trial in the first half of 2025. Sixteen experienced open-source developers, 246 real issues in repositories they had maintained for an average of five years, each task randomly assigned to allow or forbid AI tools.
The developers predicted AI would make them 24 percent faster. Measured, they were 19 percent slower. Afterwards, having lived through it, they still estimated that AI had sped them up by 20 percent.
METR treats the number as a snapshot of early-2025 tools and published new data on late-2025 tools in February 2026, so nobody should quote the 19 percent as a permanent property of AI coding. The durable finding is the gap between the two figures. Skilled professionals could not perceive the cost of verification even while paying it, on their own code, in their own repositories.
What it looks like when the checks are already there
Spotify published the other side of this in June 2026, under the title Coding Is No Longer the Constraint. More than 99 percent of their engineers use AI coding tools every week, 94 percent report being more productive, and pull request frequency is up 76 percent. Their background agent, Honk, has contributed to a Fleet Management system that has merged more than 2.5 million automated maintenance pull requests, most of them auto-merged with no human in the loop.
Auto-merged. No human. At a company with a real production system and paying subscribers.
That number only makes sense when you look at what decides whether a change survives. Honk can run builds in their CI environment across multiple operating systems to check that its own changes are correct. Their lint system and internal standards act as live guardrails, so when the agent uses a pattern that does not fit the infrastructure, it gets told immediately and corrects itself. Nobody at Spotify is trusting the model. A machine-run oracle decides, and the volume is possible because that oracle costs almost nothing per run.
There is a second finding in that post which matters more than the adoption statistics. Agent performance is measurably worse in their fragmented codebases than in their standardised ones. The value they get from AI is determined by the environment they put it in, not by the model they chose. They spent years building Backstage and Fleet Management before agents existed, and that investment is what the current numbers are sitting on.
And they are honest about where the pressure went. Seventy-six percent more pull requests means 76 percent more to review. Their own summary is that the bottleneck has moved from coding to decision-making, and that they are now working out where human judgement should be spent: auto-merge what is safe, review where it matters.
Which is the whole argument in one company’s experience. Software is the easy case. Code comes with a free oracle, Spotify built more oracles on top of it for a decade, and the constraint still moved to human understanding rather than disappearing.
So it is worth asking what happens in domains where no oracle came with the problem.
The second case: invoices in the ERP
Invoice processing is one of the few enterprise AI applications that reliably reaches production. The usual explanation, that documents are an easy problem, is wrong. Scanned invoices from 400 suppliers in six languages are not easy.
It works for the same reason Spotify works. Verification is cheap and the oracles were already in the building.
The document checks itself. Line items sum to net, net plus VAT gives gross. Arithmetic costs nothing and catches a useful share of extraction errors on its own. The ERP then supplies three more references: the purchase order says what was ordered at what price, the goods receipt says what arrived, and the supplier master holds the bank details. Three-way match is an independent verification layer that has been standard practice for decades.
The economics show up in the benchmarks. Ardent Partners’ Accounts Payable Metrics That Matter 2025 puts the average touchless processing rate at 32.6 percent and best-in-class at 49.2 percent, with exception rates of 14 percent and 9 percent respectively. Touchless means the invoice arrives, is extracted, matched, approved and posted with nobody touching it.
Read those numbers the right way round. Best-in-class does not mean a better extraction model. It means a better exception design. What separates the top performers is how small a share of documents needs a human, and PO coverage and supplier master data quality drive that far more than the AI does.
When extraction is free, checking becomes the cost structure
Fifty thousand documents a month at ten seconds of review each is about 140 hours, roughly one person doing nothing else all month. Ten seconds is optimistic. So the pressure to reduce review is immediate, and it usually arrives dressed as an accuracy figure.
The problem is that the errors are not spread evenly across the fields. They concentrate in the ones that move money:
- payment sent to an IBAN that is not the one in the supplier master
- a credit note read as an invoice, with the sign flipped
- 1.234,56 read as 1234.56
- invoice date, delivery date and due date swapped in the header block
- a duplicate paid twice because the invoice number came out of the order number field
- the wrong VAT treatment on an intra-community supply, which is a tax exposure rather than a bookkeeping error
Every one of those produces clean, well-formed, confident output. Nothing throws an error. This is DORA’s observation about uncertainty signalling, applied to finance rather than to code.
So the engineering goal is not a more accurate model. It is a cheaper check. Two things follow.
You do not route documents for human review based on model confidence. Token probability is not correctness, and treating it as a risk score fills the queue with hard-but-correct documents while the confidently wrong IBAN goes straight through. Route on oracle disagreement instead: the arithmetic does not close, the bank details differ from the master record, no goods receipt exists. Everything else posts automatically. That is Spotify’s auto-merge rule, written for accounts payable.
And the number you manage changes. Field accuracy is a vendor’s figure. Touchless rate at a stated field accuracy on payment-relevant fields, with a stated exception rate, is a figure a CFO can hold someone to, and a clause you can put in a contract.
The third case: no oracle at all
Now take the same shape of task somewhere the checks are not lying around.
Give a vision-language model a Sentinel-2 scene and ask what changed. You get a fluent, plausible paragraph. Now try to verify it. No CI job turns red. No arithmetic closes. There is no purchase order for a field in Lower Saxony. And there is no analyst who can glance at the tile and simply know, which is why anyone wanted the automation in the first place.
The research community has measured how far that goes wrong. VRSBench, a benchmark of nearly 30,000 satellite images built with more than a thousand hours of expert human verification, found that models are reasonably good at saying what kind of object is present and much weaker at quantity and direction. Overhead scenes break the assumptions these models were trained on: objects are sometimes ten pixels across, and spatial reasoning happens in a coordinate system that has no analogue in internet photographs.
A more recent benchmark on low-level perception in remote sensing describes the failure mode as a fluency illusion: models produce descriptions that read as confident and visually grounded while misidentifying the actual degradation type or misjudging its severity. Confusing SAR speckle with Gaussian noise, for instance. The output is not obviously hallucinated. It is precise-sounding and wrong.
There is a second problem underneath, and it is one of pipeline design rather than model quality. Sentinel-2 delivers thirteen calibrated spectral bands. To feed a vision model you choose three, apply a percentile stretch, and hand over an 8-bit RGB image. Widen the clip by one point at each end and the description of the scene changes. At that point the colormap is a model parameter, and nobody is tracking it as one.
Sentinel-2 L2A, 13 August 2026, farmland in Lower Saxony centred on 52.86° N, 9.31° E, about 31 by 20 km at 20 m per pixel. This is the reduction described above, carried out: three of the thirteen bands (B04, B03, B02), each stretched linearly between its 2nd and 98th percentile and quantised to 8-bit RGB. Those bounds — red 180–2324, green 280–1642, blue 168–1200 in raw reflectance counts — are a choice made by whoever built the pipeline, not a property of the scene. Contains modified Copernicus Sentinel data 2026.
The model also has no ground sample distance, so it cannot separate vehicles from shipping containers by size. It has no radiometry, so it can report that vegetation looks stressed but cannot tell you whether NDVI is 0.31 or 0.62, and only one of those supports a decision. Two acquisitions with different sun angles read as change that never happened.
Here the oracles have to be built or bought: spectral indices and atmospheric correction as quantitative reference, radar as an independent modality whose failure modes do not correlate with the optical pipeline, ground measurements where they exist. Running a second vision model and comparing is not verification. Two models trained on overlapping data agree for reasons that have nothing to do with the scene.
The architecture that works keeps the physics in the pipeline. Instrumented methods compute the quantities and carry their uncertainty. The language model sits on top of those products as an interface, so an analyst can ask questions and get explanations. That is a valuable role. It is not the measurement role, and most demos confuse the two.
Two questions before you fund a use case
Across those three cases there is a test you can apply in a portfolio review, before anyone builds anything.
What does one verification cost? And does an independent oracle already exist in our systems, or do we have to create one?
Code at Spotify: milliseconds, and yes, because CI was already there. Invoices: seconds, and yes, because the ERP was already there. Satellite imagery: expensive, and no. Most proposals sit somewhere between, and almost nobody scores them on this axis. They get scored on model capability, which is the one input that stopped being scarce.
Verification capacity is the real ceiling
If the check is what limits throughput, the check is what deserves the budget. These are the line items that rarely appear in an AI business case.
An evaluation set that is stratified rather than random. Random samples over-represent clean, easy inputs and flatter the model. The set has to be weighted toward the scanned, the skewed, the multi-language, the hazy winter scene.
A locked, versioned regression suite. Hosted models change underneath you without a release note. Without a suite that re-runs on a schedule, you learn about a silent degradation from a customer.
Abstention as a first-class output. A system that cannot say “I could not read this” forces every one of its errors into the downstream process. DORA’s finding about uncertainty signalling is a product requirement, not a research curiosity.
Provenance, so a human check takes five seconds instead of five minutes. Every extracted value should point back to the evidence it came from: the box on the page, the band and date of the scene.
None of these make the model better. All of them make the check cheaper, which is the same thing as more capacity. Spotify’s version of this list took years to build, and it is the reason their agent numbers are worth anything.
What you cannot verify, you cannot leave
There is a commercial reason to care about this beyond error rates.
If nobody on your side understands how the output is produced or how you would know it was wrong, you cannot write the specification for the next version, you cannot evaluate a competing quote, and you cannot move. The switching cost is not in the integration. It sits in the fact that your own organisation has no independent way to judge the thing it is buying.
That is a slower and more expensive problem than a bad extraction, and it does not show up on any dashboard until you try to change supplier.
Where this leaves a decision
Spotify’s title says coding is no longer the constraint, and their own conclusion is that the constraint moved to decision-making rather than vanishing. DORA says the same thing from the other direction: adoption is near universal and stability has not caught up. The bottleneck is now sitting in testing, review and judgement, all of which were sized for a world where producing the work was the slow part.
Score your AI portfolio on the cost of checking rather than on the capability of the model. Fund the oracles before you scale the generation. Where no oracle exists, building one is the project, and pretending otherwise is how a convincing demo becomes a system nobody can defend.
If you are deciding what to build, what to buy, or whether a vendor’s accuracy number means anything, that is the conversation we have with clients. Book a strategy and architecture review.
Sources
- Google Cloud: Announcing the 2025 DORA report
- DORA: Balancing act — the tensions of AI-assisted development
- METR: Measuring the impact of early-2025 AI on experienced open-source developer productivity
- Spotify Engineering: Coding Is No Longer the Constraint
- VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding
- A benchmark for low-level perception in remote sensing vision-language models
- Satellite image: Sentinel-2 L2A scene
S2A_32UND_20260813_1_L2A, tile 32UND, acquired 13 August 2026, obtained from the Earth Search STAC catalogue on AWS Open Data. Contains modified Copernicus Sentinel data 2026.