Analysis · Coding agents
Assign the issue, get a pull request. What does an agent’s PR really cost?
The newest way to use AI in software is to stop watching it. You assign an issue to a cloud agent, it works in its own sandbox, and a pull request appears. Codex, Copilot’s cloud agent, Devin, Cursor and Claude Code all offer a version of this. Two September studies of real agent pull requests let us ask a sharper question than whether it works: what does the PR cost by the time it is really done?
Nythrex team · 10/28/2026
In brief
7-minute read · Updated 28 October 2026
- Quality varies by agent. In 37,623 pull requests across 2,807 repositories, Codex PRs were reverted about half as often as human PRs, Devin’s more often.
- Merged is not finished. Merged agent PRs needed follow-up fixes at 1.62 times the odds of human PRs in the same repositories.
- Agents clean up after themselves. 69.6% of verified fixes to agent merges came from the same agent.
- The real cost of an agent PR is the run, the CI, the human review and every fix that follows.
What the research found
The first study, “Not All Agents Are Equal”, labelled the provenance of 37,623 pull requests from five agents and a human baseline, opened between December 2024 and July 2025. The differences between agents were larger than the difference between agents and people.
0.63
Odds ratio for a security smell in pooled agent code versus human code, mostly fewer hard-coded credentials.12.6 hours
Median wait for the first human review on Claude Code PRs, the longest of the five agents.Most reviews
Copilot PRs drew the most human reviews and change requests.
The follow-up fix problem
The second study, “Who Finishes the Job?”, followed 6,774 merged agent PRs and 5,044 human PRs from the same repositories, all projects with at least 500 stars. Agent merges attracted verified follow-up fixes at 1.62 times the odds of human merges. Most of those fixes came from agents too: 69.6% from the same agent, and 76.4% of fix PRs were agent-authored in every commit.
That is reassuring and expensive at once. Agents do finish their own work, but each fix is another agent run, another CI pass and another review. None of it is visible if you only look at the PR that was merged first.
An agent pull request is not finished when it is merged. It is finished when nobody has to touch it again.
Anatomy of an agent PR’s cost
Put the findings together and the cost of an autonomous pull request has five parts. Only the first one shows up on the AI invoice.
| Stage | What it costs | Where it shows up |
|---|---|---|
| Agent run | Tokens, credits or subscription allowance | AI provider invoice |
| CI iterations | Minutes for every attempt, including failed runs | CI bill |
| Human review | Reviewer time and hours of waiting | Nowhere: it is payroll |
| Follow-up fixes | Another run, more CI, another review | Spread across later changes |
| Reverts | All of the above, plus incident time | Rarely linked to the original PR |
How to run agents without losing the thread
- 1
Scope tasks tightly
Cloud agents are tuned for well-defined issues. Vague tasks produce longer runs, more retries and more fixes. - 2
Compare agents on your own repositories
Revert and fix rates differ widely between agents. Your codebase is the benchmark that matters. - 3
Link fixes back to the original change
A fix PR is part of the cost of the PR it fixes. Without the link, every agent change looks cheaper than it was. - 4
Budget reviewer time
Agents can open pull requests faster than people can review them. The review queue is now a cost centre. - 5
Count what never merged
Closed and abandoned agent PRs consumed tokens and CI. They belong in the total.
Where Nythrex fits
Nythrex treats task, AI session, commit, pull request, CI and deploy as one unit of work, so the cost of a change includes the CI it triggered and the sessions that never shipped. Named rules point at money that produced no result, with the records and arithmetic behind each amount.
Sources
- arXiv 2609.17598 — Not All Agents Are Equal: code quality and post-merge maintenance across five autonomous coding agents
- arXiv 2609.26847 — Who Finishes the Job? Follow-up fixes and commit authorship on AI coding agent pull requests
- Firecrawl — Best AI coding agents in 2026: harness, cost and accuracy compared
Frequently asked questions
Are pull requests from AI coding agents worse than human ones?
It depends on the agent. In a study of 37,623 pull requests, Codex PRs were reverted at 6.1% versus 11.5% for humans, while Devin PRs were reverted at 14.5%. Another study found merged agent PRs needed follow-up fixes at 1.62 times the odds of human PRs.
Who fixes agent pull requests after they are merged?
Mostly agents. In a study of 6,774 merged agent PRs, 69.6% of verified fixes came from the same agent that wrote the original change.
What does an agent pull request really cost?
The agent run is only one part. Add the CI minutes for every attempt, human review time, follow-up fixes and any reverts. Linking fixes back to the original change is what makes the full cost visible.
Read next
- The CI bill in the agent era: what GitHub’s Actions pricing reset tells engineering teamsGitHub cut hosted runner prices by up to 39% in January and postponed a planned $0.002-per-minute charge on self-hosted runners. Coding agents that iterate through CI will make both decisions matter more.Read
- Faster code, same delivery? What AI is really doing to the pace of software developmentAI coding tools made writing code dramatically faster. Shipping software did not speed up by the same amount. An essay on where the time actually goes, why the bottleneck moves, and why the world is not accelerating evenly.Read
- How Nythrex works: from AI session to productionHow Nythrex links the task, AI session, commit, pull request, CI run, deployment and cloud into one unit of work with its own cost and the basis for every figure.Read
Stop guessing.
Start seeing.
No card and no subscription: paid plans have not launched yet.