The Uop Cache's Alignment-Sensitive Fill Behavior: Why 32-Byte-Aligned Loops Fit and Misaligned Ones Don't

2026-09-02

The decoded uop cache (DSB) on Intel CPUs stores pre-decoded micro-ops indexed by 32-byte-aligned fetch windows in the original instruction stream. Each such window maps to at most 3 uop cache lines, with each line holding up to 6 uops. That means a single 32-byte region of code can cache at most 18 uops. Cross that boundary and you're in the next set — with its own separate 3-way, 18-uop budget.

The trap: the CPU allocates cache lines as it fills them from decode, and a line ends whenever the fetch window ends, a branch is taken, or the 6-uop line fills up. If your hot loop straddles a 32-byte boundary awkwardly, you can fragment uops across sets so badly that the loop no longer fits, kicks out to the legacy decoder, and loses ~50% of front-end bandwidth (from 6 uops/cycle DSB delivery down to 4 uops/cycle legacy decode, plus the pre-decode length-finding penalty).

Real-world example: A well-known case from Intel's optimization manual and confirmed by Agner Fog's microarchitecture guide: a 28-uop tight loop that fits comfortably when aligned to a 32-byte boundary (one set: 28 uops needs 5 lines... wait, that exceeds 3). Correction: the loop must fit in at most two adjacent 32-byte sets to remain fully DSB-resident. Add a single byte of code before it via a linker change and the loop shifts, now spanning three sets — the last set only has room for a fragment, the fragment gets evicted, DSB delivery drops, and the loop runs 15-25% slower. Nothing else changed. This is why -falign-loops=32 exists in GCC and why performance-critical inner loops often get NOP-padded.

Rule of thumb: If your hot loop is under ~50 uops, align its entry to a 32-byte boundary. Count roughly 1 uop per simple x86 instruction (2-4 for complex ones like memory-operand ALU ops with immediates). If uops_delivered.dsb / uops_delivered.total drops below 80% in perf counters, you're bleeding front-end bandwidth to alignment.

The deeper lesson: the uop cache isn't a magical decoded-instruction pool — it's a set-associative cache indexed by the original x86 fetch address. Every constraint of a physical cache (associativity, line size, fill boundaries) still applies, just at the uop granularity. Move your code by one byte and you can change which set it hashes into.

Key Takeaway: The uop cache is indexed by 32-byte-aligned x86 fetch windows with only 3 lines per set, so shifting hot code by a single byte can fragment it across sets and cost you nearly half your front-end bandwidth.

All newsletters