The Store Buffer's Store-to-Load Forwarding Store Coalescing Limit: Why Adjacent Byte Stores Don't Always Fuse Before Forwarding

2026-09-11

When your CPU executes a store, the value doesn't go straight to cache — it sits in the store buffer, a small structure (56 entries on Skylake, 72 on Ice Lake, 128+ on modern Zen) that holds committed-but-not-yet-retired stores. A younger load can grab the value directly from the store buffer via store-to-load forwarding, skipping the cache entirely. But there's a subtle limit that trips up hand-tuned code: the store buffer generally does not coalesce adjacent stores before forwarding.

Consider what happens when you write four bytes to consecutive addresses:

Logically, the load could assemble its result from four adjacent store buffer entries. In practice, it can't. Store-to-load forwarding hardware handles exactly one older store overlapping the load. When multiple stores together cover the load's range, the forwarding logic gives up, stalls the load, and waits for those stores to drain to L1 before re-issuing the load from cache. That drain-and-replay penalty is typically 10–20 cycles on Intel, versus ~5 cycles for a successful forward.

Why doesn't the hardware just merge them? Coalescing requires: age-ordering the stores, checking address overlap across N entries, verifying no intervening fence, and assembling bytes with correct byte-enables — all inside the forwarding critical path, which must complete in a single cycle. Doing this for N=4 stores would blow the timing budget. So the hardware punts.

Concrete real-world example: A memcpy loop written with byte stores for a 4-byte tail (common in older glibc fallback paths) followed by a 4-byte load of the same region can run 3–4× slower than the same loop written with a single 32-bit store. Modern glibc's __memmove_avx_unaligned deliberately uses the widest store that fits the tail specifically to avoid this pathology.

Rule of thumb: If a load reads N bytes from a region, ensure the single most recent store covering that region is at least N bytes wide and aligned to include the entire load range. Then forwarding hits the fast path (~5 cycles). Otherwise, budget for the drain penalty: ~15 cycles minimum, plus L1 access time.

Some newer designs (Zen 4, Apple's cores) can coalesce two adjacent stores in narrow cases, but treating this as reliable is a mistake — the fast path only exists when one store fully covers the load.

Key Takeaway: Store-to-load forwarding matches exactly one older store to the load, so a wide load fed by multiple narrow adjacent stores stalls until those stores drain to L1 — always make the covering store as wide as the load.

All newsletters