The Working Set of a Coding Agent: Coherence Debt in Repository-Scale Tasks

2026-08-18

Authors: Bardia Mohammadi, Lars Klein, Aman Chadha, Akhil Arora

ArXiv: 2608.16630v1

PDF: Download PDF

When an AI coding agent works on a real repository — not a leetcode puzzle, but something with dozens of files, tests, imports, config, and migration rules — it has to keep a lot of facts straight simultaneously. A function signature in one file has to match its callers in another. An import has to actually exist. A migration has to line up with the schema. The agent's context window is finite, so it can only hold so many of these facts at once. What it can't hold, it has to remember from training data (what the authors call "parametric memory").

This paper introduces a clean way to think about that tension. Every edit an agent makes depends on some required facts. Each fact either comes from recent context (something the agent just read) or from parametric memory (something baked into the model's weights during training). The facts that come from neither — the ones the agent should know but doesn't — accumulate as what they call coherence debt. That debt is what causes the agent to hallucinate an import, break a test, or introduce a subtle inconsistency three files away from where it's editing.

To test this, the authors run seven different models across five agent harnesses. They deliberately give and withhold each channel — sometimes stripping context, sometimes using APIs the model has never seen (so parametric memory is useless), sometimes injecting deliberate faults. The unsurprising headline: no model completes a task when both channels are unavailable. But the interesting findings are in the middle ground — how much each channel compensates for the other, and where the failure modes hide.

Why is this framing useful? Most current evaluations of coding agents just report pass/fail on benchmark tasks. That tells you whether the agent worked, but not why it failed when it did. Was the context window too small? Did the model hallucinate a function that doesn't exist? Was the repo just too weird for its training data? Coherence debt gives you a diagnostic vocabulary. It reframes "the agent got confused" as "the agent needed fact X, and neither channel supplied it."

The practical implication is significant for anyone building agent harnesses. Making the context window bigger isn't automatically the answer — you have to give the agent the right facts, and the right facts depend on which ones the model already knows. For an unfamiliar internal API, aggressive retrieval matters. For a well-known open-source library, you can lean on parametric memory and save tokens for the parts that are actually novel.

Why it matters: It gives us a principled way to reason about why AI coding agents fail on real repositories, turning "the model got confused" into a measurable gap between what the agent needs to know and what its two memory channels actually supply.

All newsletters