When a load misses L1, it doesn't just wait somewhere in the abstract — it occupies a specific piece of hardware called a Line Fill Buffer (LFB, also called MSHR in academic parlance) from the moment the miss is detected until the cache line arrives, gets written into L1, and the load's result is delivered to the register file. Everything about your memory-bound performance ultimately reduces to how many LFBs you have and how long each one is held.

Intel Skylake through Golden Cove has 12 LFBs per core. AMD Zen 4 has around 22 miss-handling slots. That number is the hard ceiling on outstanding L1 misses. Once every LFB is occupied, the next load that misses L1 stalls in the load queue — it can't even issue a memory request, because there's nowhere to track it.

The lifetime matters as much as the count. An LFB is allocated when the load misses L1, and freed only after: the request travels to L2 (12 cycles), possibly to L3 (40 cycles), possibly to DRAM (200+ cycles), the line comes back, gets written into L1, and the data is forwarded to any loads waiting on it. For a DRAM miss, that's ~250 cycles of LFB occupancy per miss.

The math that governs memory-bound code: Little's Law says throughput = concurrency / latency. With 12 LFBs and 250-cycle DRAM latency, your maximum sustained miss rate is 12/250 = 0.048 misses per cycle, or roughly one miss every 21 cycles. At 4 GHz with 64-byte lines, that's 12 GB/s per core — regardless of how much DRAM bandwidth your DIMMs can theoretically deliver.

Concrete example: a pointer-chasing linked list traversal. Each load depends on the previous one, so only one LFB is used at a time. You get 1/250 misses/cycle = 256 MB/s — the other 11 LFBs sit idle because there's no memory-level parallelism to exploit. Compare that to a streaming array sum: the prefetcher issues 12 concurrent misses, all 12 LFBs stay busy, and you hit the 12 GB/s ceiling.

This is why reducing latency and increasing MLP matter more than raw bandwidth for a single thread. Adding a second DIMM channel doesn't help pointer chasing. Restructuring the data to enable parallel misses does.

Rule of thumb: Peak per-core DRAM bandwidth ≈ (LFB count × line size × clock) / DRAM latency in cycles. If you're not close to that number, you have MLP to unlock. If you are, only latency reduction or more LFBs will help.