Q&A · AI ROI
Our CFO asked whether AI coding tools pay for themselves. What do I actually show them?
One of the questions we hear most often from CTOs and engineering leads at 20–200-person teams. Here is our answer, and the evidence behind it.
Nythrex team · 10/29/2026
The question
Asked by an engineering lead · the way we usually hear it
“We pay every month for Copilot seats, Claude Code and a couple of API keys. In the budget review the CFO asked one simple question: is it paying for itself? I said the team feels faster. That was not an answer. What should I bring next time?”
Short answer
Leave adoption charts and impressions at home. Bring the cost of shipped work: what AI, CI and cloud cost per change that reached production, by project, before and after. How fast people feel is the weakest evidence you have — the best-known controlled study found developers felt faster while they were measurably slower.
Why “the team feels faster” doesn’t survive the meeting
In July 2025 METR ran a randomised trial with 16 experienced open-source developers working on 246 real issues in repositories they knew well. With AI tools allowed, they took 19% longer. Before the study they expected AI to speed them up by 24%; afterwards they still believed it had sped them up by about 20%.
That result is dated, and METR says so. In February 2026 it wrote that developers are probably sped up more now than in early 2025, but that its newer data is only very weak evidence of how much — partly because many developers no longer agreed to work without AI at all. Both findings point the same way: self-reported speed is not a measurement.
Company-level numbers are no kinder. MIT NANDA’s preliminary 2025 report on generative AI in business found that 95% of the organisations it studied saw no measurable return from their initiatives. It is a small, early sample, and it covers far more than coding tools — but it is exactly the headline your CFO has read.
What a CFO will accept
| What you could show | How strong it is | Why |
|---|---|---|
| Seats bought, adoption rate | Weak | Shows money spent, not what it bought |
| “Developers feel faster” | Weak | Perception and measured speed diverged in the METR trial |
| AI cost per change shipped, by project | Strong | Ties spend to delivered work |
| Share of AI spend on work that never shipped | Strong | Shows waste without blaming anyone |
| Delivery trend next to spend trend | Strong | Answers “more money, more output?” |
What we’d bring to the next review
- 1
Agree on the unit
A change that was merged and deployed. Not a prompt, not a seat, not a person. - 2
Take a baseline
Two or three months of cost per shipped change by project, so the next quarter has something to be compared with. - 3
Attach AI to the work
Link AI sessions to the repository and pull request they worked on. Spend that can’t be linked stays visible as its own line. - 4
Show ranges and their basis
Subscription usage is an estimate; say so. A CFO trusts a labelled estimate more than a precise-looking number that turns out to be a guess. - 5
Repeat quarterly
ROI is a trend, not a verdict. The same method every quarter makes the trend believable.
Where Nythrex helps — and where it doesn’t
What it does
Shows the cost of each change from task to AI session, commit, pull request, CI and deploy, by project, team and period. Claude Code, Codex and GitHub Copilot connect alongside your repositories and tracker. Every figure shows its basis; subscription usage is marked as an estimate from the price list, and an unknown price is never shown as zero.What it doesn’t
It doesn’t tell you the revenue impact of AI, and it doesn’t rate people’s productivity or compare them with each other. It gives you the cost side of the ROI conversation, with evidence.
Frequently asked questions
Do AI coding tools make developers faster?
It depends on the work and the period. METR’s July 2025 randomised trial found experienced developers took 19% longer with AI while believing they were 20% faster; in February 2026 METR said developers are probably sped up more now, but its data is only weak evidence of how much.
How do I measure the ROI of AI coding tools?
Measure the cost side against delivered work: AI, CI and cloud cost per change that reached production, by project, with a baseline and a quarterly trend. Show the share of AI spend that never led to shipped work separately.
Should I show adoption rates to the CFO?
Only as context. Adoption shows that money was spent, not what it bought. Cost per shipped change and the delivery trend are the evidence a CFO can act on.
Read next
- Measure AI, not the people using itLeaders need to show what AI is worth, and the easiest metric is usage per person. Developers fear it, it pushes costs up, and Meta has already dropped it from reviews. The better unit is the change, not the engineer.Read
- The monthly engineering report that eats a day, and why nobody trusts itThree in four engineering managers must report on team performance, and fewer than half believe their metrics reflect real value. The problem is not effort: it is numbers assembled by hand from five tools with no source behind them.Read
- How Nythrex works: from AI session to productionHow Nythrex links the task, AI session, commit, pull request, CI run, deployment and cloud into one unit of work with its own cost and the basis for every figure.Read
Stop guessing.
Start seeing.
No card and no subscription: paid plans have not launched yet.