When an AI writes code, it typically works like a novelist typing without a spellchecker: it produces the whole draft, then someone runs it through a compiler afterwards to see what broke. If the code has type errors, undefined variables, or borrow-checker violations (a Rust specialty), you either throw it away or feed the errors back and ask for a rewrite. This is wasteful, especially for languages like Rust where the compiler enforces strict rules about memory ownership that trip up even experienced humans.

The authors propose generative compilation: what if the compiler ran alongside the AI, checking each token as it was generated, and steering the model back on track the moment things started to go wrong?

Two existing approaches almost solve this but fall short:

• Post-hoc feedback — running the compiler after generation is finished. Too late; the model has already committed to a broken structure.
• Constrained decoding — filtering out invalid tokens during sampling. This works for grammar (matching brackets, valid keywords) but chokes on semantic rules like "this variable was already moved and can't be used again." Implementing those checks means reinventing large chunks of the compiler, and it requires white-box access to the model's internals.

Their trick is to sit between these two: let the AI generate freely for a bit, but continuously invoke the real compiler on the partial output as it grows. When the compiler flags an error, the system rolls back to just before the mistake and nudges the model with the error message, so it retries that stretch with better information. Crucially, this works with black-box models — you don't need to modify the LLM itself, just wrap its API.

Because they use the actual compiler rather than a reimplementation, they get the full richness of Rust's semantic checks (lifetimes, traits, borrow checking) for free. And because feedback happens mid-generation rather than after, the model doesn't have to unwind a whole broken function to fix a mistake made three lines in.

The paper shows this approach materially improves the fraction of Rust programs that compile on the first try, without needing model fine-tuning or bespoke grammars for every semantic rule.