The Uop Cache's Way-Prediction Bypass: How CPUs Skip Tag Checks on Sequential Fetch

2026-08-31

The uop cache (decoded instruction cache) is set-associative like any other cache — 8 ways per set on Skylake, for example. Every fetch normally has to check tags across all ways to find the right decoded uops. That tag comparison burns power on every single cycle the front-end runs, and the uop cache runs a lot.

The trick modern CPUs use: way-prediction with a sequential fetch bypass. When the front-end reads a uop cache line, the line itself records which way holds the next sequential line. On the next cycle, if the fetch address matches the predicted successor, the CPU reads directly from that way — no tag comparison at all. It's essentially a linked list embedded in the cache: line N points to the way holding line N+1.

This is why straight-line hot code hits the uop cache at near-zero power cost, but taken branches are expensive even when they hit. A taken branch breaks the sequential prediction chain, forcing a full tag lookup across all ways of the target set. That's part of why the uop cache has a "3 uop lines per taken branch per cycle" limit — the way-prediction bypass only works for the fall-through path.

Concrete example: Consider a hot loop with 10 uop cache lines and no internal taken branches. First iteration: cold, misses, has to decode. Second iteration: the way-prediction pointers are populated, so 9 of the 10 fetches bypass tag checks entirely — only the loop-back branch pays the full lookup cost. Now compare that to code peppered with conditional branches: even if every branch is predicted correctly and every target hits the uop cache, every taken branch pays the full tag comparison cost because the sequential chain broke.

Rule of thumb: Front-end power scales roughly as 0.2 × (baseline) + 0.8 × (taken_branch_fraction). If your hot loop has one taken branch per 30 uops (~3%), the front-end runs at ~22% of full tag-check power. If it's one taken branch per 5 uops (~20%), you're at ~36% — nearly double the front-end power for the same uop throughput.

This also explains a weird measurement artifact: on Intel UOPS_ISSUED.ANY can be identical between two versions of code, but CORE_POWER.LVL0_TURBO_LICENSE or package power differs by 5-10%. The extra power isn't going to execution — it's going to tag comparisons in a front-end that lost its way-prediction chain.

See it in action: Check out Silicon to Token: How the Hardware of Machine Learning Actually Works by Vector Meridian to see this theory applied.
Key Takeaway: The uop cache saves power on sequential fetch by chaining way pointers between lines, so taken branches cost energy even when they hit — straight-line code isn't just faster, it's dramatically cooler.

All newsletters