Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset

2026-06-13

Authors: Mahmoud Abujadallah, Ali Arabat, Mohammed Sayagh

ArXiv: 2606.13468v1

PDF: Download PDF

If you've used Copilot, Cursor, Devin, or Claude to open a pull request that "fixes" a bug, you've probably had the experience of watching a human reviewer close it without merging. This paper puts a number on how often that happens — and it's startling. Looking at the AIDev dataset (a collection of real pull requests opened by AI coding agents across open-source projects), the authors found that 46.41% of agent-proposed fixes get rejected. Nearly half. Every one of those rejections represents wasted compute, wasted CI runs, and — most expensively — wasted human reviewer time spent reading, testing, and ultimately discarding code.

The authors aren't just complaining about the number. They dig into why these PRs fail, categorizing the failure modes across the four major agents. A few patterns jump out:

The key insight is that "the code runs" is a much weaker bar than "the maintainer will merge it." Agents have largely optimized for the former — generating syntactically valid, test-passing diffs — while the actual bottleneck for adoption is a fuzzier social and architectural fit. A pull request isn't just code; it's a proposal that has to make sense within a project's history, conventions, and the maintainer's mental model of what the codebase should become.

This has practical implications. If you're a maintainer, the paper effectively predicts which agent-generated PRs are worth your time. If you're building coding agents, it's a roadmap of what to fix next: better issue comprehension, tighter scoping, and stronger sensitivity to project conventions matter more than raw code-generation quality. And if you're a developer using these tools, it's a reminder that the agent's confident-looking PR has roughly coin-flip odds of being accepted upstream — so reviewing it yourself before pushing is still very much your job.

Why it matters: Nearly half of all AI-agent pull requests get rejected, exposing a costly gap between code that compiles and code that maintainers will actually merge — and pointing to where the next generation of coding agents needs to improve.

All newsletters