N-Version Programming with Coding Agents

2026-06-19

Authors: Javier Ron, Benoit Baudry, Martin Monperrus

ArXiv: 2606.20158v1

PDF: Download PDF

Back in 1986, Knight and Leveson ran a famous experiment in software engineering. They asked 27 programmers to independently write the same program from the same spec, hoping that if you ran several copies side-by-side and took a majority vote, you'd get a system more reliable than any single version. The disappointing result: the programmers tended to make the same mistakes on the same tricky inputs. Independent humans weren't actually that independent. This experiment essentially killed "N-version programming" as a practical reliability technique for decades.

This paper asks an obvious but timely question: what if we redo the experiment, but with AI coding agents instead of humans? Agents don't have shared educational backgrounds or office-water-cooler bias. You can vary the underlying LLM, the agent framework, and even the target programming language. Maybe that's enough diversity to make voting actually work.

The authors took the same spec Knight and Leveson used — the Launch Interceptor Program, which decides whether a missile defense system should fire — and generated 48 implementations using different combinations of agents, models, and languages. They then hammered each implementation with one million randomized test inputs and compared results.

The finding: diversity actually shows up this time. Different agent/model/language combinations produce meaningfully different failure modes. Where the 1986 humans clustered their bugs around the same hard cases, the AI agents spread their failures around more, which is exactly the property that makes majority voting useful. The variations across implementation languages (e.g., Python vs. Java vs. Rust) seemed to matter as much as variations across models.

A few things to keep in mind:

Still, the result is a quiet but significant update to a 40-year-old conclusion. Cheap, parallel code generation changes the economics of fault-tolerant software: it's now plausible to ship five versions of a critical component and vote between them, where in 1986 you'd have had to pay five teams of engineers.

Why it matters: AI coding agents may finally make N-version programming — a long-abandoned reliability technique — practical, because they fail differently from each other in ways human programmers never did.

All newsletters