2026-05-20
If you've ever tried to put an LLM into a real production system, you've felt the awkward seam: the model is a creative, unpredictable thing that generates plausible-sounding text, while the surrounding software is rigid, deterministic, and expects exact inputs. Most teams duct-tape across that seam with retries, JSON parsers, validators, and human approvals — but nobody really names the seam itself or treats it as a thing to design.
This paper does exactly that. It gives the seam a name: the stochastic-deterministic boundary, or SDB. And it argues that the SDB is the single most important architectural concept in any production LLM agent — more important than the model choice, the prompt, or the framework.
The author defines the SDB as a four-part contract:
The insight is that almost every reliability problem in production agents comes from a poorly-specified version of one of these four pieces. A weak verifier means hallucinations slip through. A commit step that happens before verification means you can't roll back. A vague reject signal means the LLM keeps making the same mistake. Teams that get burned by their agents usually skipped one of these.
From this primitive, the paper builds a methodology for selecting and composing runtime architecture patterns — things like guarded tool use, plan-then-execute, multi-agent review, and human-in-the-loop. Instead of treating these as competing trends, the paper shows they're all just different ways of arranging proposers, verifiers, and commits. That reframing lets you reason about which pattern fits your situation: how reversible are your actions, how expensive is verification, how much latency can you tolerate?
The practical payoff is that engineering teams stop arguing about "should we use LangGraph or build our own?" and start asking the right questions: where exactly does stochastic output become a real-world action, and what's checking it at that exact moment? That's a much more productive conversation, and it maps cleanly onto things SREs and architects already know how to reason about — transactions, idempotency, validation boundaries.
