2026-06-11
Authors: Hao-Nan Zhu, Goodness Ayinmode, Cesar A. Stuardo, Haryadi S. Gunawi
ArXiv: 2606.11815v1
PDF: Download PDF
Imagine you build a distributed system — say a database or a message queue — and it works beautifully in testing with 10 nodes. You ship it. Six months later, a customer runs it on 1,000 nodes and everything melts down: latency spikes, memory blows up, the cluster won't even finish starting. The code is "correct" in the sense that nothing crashes at small scale, but it has a scalability fault — a bug that only wakes up when the system gets big.
This paper is the first systematic study of those bugs. The authors went through 444 real scalability issue reports from 10 major distributed systems (think the kind of infrastructure that runs at Google, Netflix, or your favorite cloud provider) and asked: what patterns keep showing up?
A few of their findings are genuinely useful:
The second half of the paper proposes a detection approach. Instead of trying to spin up thousand-node clusters (expensive, slow), they statically analyze code looking for the recurring anti-patterns they catalogued — loops over cluster-sized collections, message fan-outs without batching, per-node state that scales with peer count, and so on. It's basically a linter that knows what scalability bugs look like at the source level, informed by hundreds of real ones.
Why this is a big deal: distributed systems testing has always had a painful gap. Unit tests catch logic bugs. Chaos engineering catches failure-handling bugs. But "works at 10, dies at 10,000" bugs have mostly been caught by customers. A taxonomy plus a detector that runs at code-review time could shift that left by years.
The honest caveat: static detection will have false positives, and some scalability faults emerge from emergent behavior no analyzer can predict. But even catching the obvious ones — the accidentally-quadratic gossip loop, the unbatched broadcast — would prevent a lot of 3am pages.
