2026-06-16
Your CPU doesn't talk to DRAM directly — it hands requests to a memory controller, which reorders them before sending commands on the DDR bus. The reason is simple: DRAM is brutally asymmetric. Reading from an already-open row costs ~15ns (a CAS), but reading from a closed row costs ~50ns (PRECHARGE + ACTIVATE + CAS). A controller that issues requests in arrival order leaves 3× performance on the floor.
The dominant policy for two decades has been FR-FCFS — First-Ready, First-Come-First-Served. The rules, in priority order:
The effect: a stream of requests like RowA, RowB, RowA, RowB, RowA gets reordered into RowA, RowA, RowA, RowB, RowB. Each cluster pays the open-row penalty once and then bursts at CAS speed.
Concrete example: a GPU running a matrix multiply generates thousands of in-flight requests per cycle. Without row-hit reordering, a typical kernel hits maybe 30% row hit rate. With FR-FCFS and a deep queue (64+ entries), that climbs to 80%+, which roughly doubles effective bandwidth on the same DRAM chips.
Rule of thumb: the row-hit speedup is tRC / tCCD, where tRC ≈ 45ns is the full row cycle and tCCD ≈ 5ns is the column-to-column delay. That's ~9× per-access — and a 50% hit rate gets you roughly halfway there in average latency.
The catch is starvation. A thread hammering one row can monopolize the bank forever, since each new request is always "row hit, ready, go." Real controllers add a cap — typically 4–16 consecutive row hits — after which they force the oldest pending request through, even if it triggers a precharge. Server controllers (Intel, AMD) layer per-thread fairness on top: PAR-BS groups requests by thread and rotates batches, so one noisy neighbor can't kill another VM's latency.
Modern controllers also predict whether to leave a row open after the last access (open-page policy, good for spatial locality) or auto-precharge (closed-page, good for random access). Some adapt per-bank based on the last few access patterns — a tiny ML problem solved with a 2-bit counter.