You already know about the µop cache and the loop stream detector. The decoded instruction cache (sometimes called the L0 instruction cache or decode cache) is a related but distinct beast — it caches the raw decoded micro-ops for arbitrary code paths, not just hot loops, and sits in front of the rename stage as a complete bypass for fetch, predecode, and decode.

The problem it solves: x86 decode is brutal. Variable-length instructions (1–15 bytes), prefixes, ModR/M bytes, and SIB bytes mean the legacy decoder needs predecode (length-marking), then 4 parallel decoders (one complex, three simple), then µop fusion logic. That's roughly 4–5 pipeline stages and ~5 watts on a modern Intel core just to turn bytes into µops. Doing it once per execution is fine; doing it billions of times for the same hot code is wasteful.

The decoded I-cache stores µops indexed by their original instruction-pointer address. On a hit, the front-end powers down predecode and the legacy decoders entirely — µops stream directly into the µop queue. Intel's Sandy Bridge had a 1,536-entry µop cache (~6 µops/line, 32 sets, 8-way). Golden Cove pushed this to 4,000 entries. AMD Zen 4's op cache holds ~6,750 µops.

Concrete example: A tight JSON parser loop of ~200 x86 instructions fits entirely in the decoded I-cache. Once warm, the legacy decoder is clock-gated. Perf counters show idq.dsb_uops (Decoded Stream Buffer hits) near 100% and idq.mite_uops (legacy decode path) near zero. Power drops measurably — Intel has cited ~10% front-end power savings on decode-bound workloads.

Rule of thumb: Aim for >80% DSB hit rate on hot code. The decoded cache lines have alignment constraints: a single 32-byte instruction window must produce ≤18 µops and cannot cross certain boundaries, or it falls back to legacy decode. This is why compilers care about 32-byte alignment for hot loops (-falign-loops=32).

Failure modes worth knowing:

• JIT code thrashes it. Generated code at fresh addresses misses every time until warm.
• Long instructions with many prefixes (think AVX-512 with EVEX prefix + mask + broadcast) consume multiple cache slots per instruction, reducing effective capacity.
• Self-modifying code triggers full invalidation — another reason JITs prefer write-once code pages.
• Branch-dense code fragments cache lines; each taken branch typically ends a cache line early.

ARM cores generally lack an equivalent because fixed-width AArch64 decode is cheap (~1 stage). This is one of the quiet architectural advantages RISC ISAs still hold — x86 needs an extra megabyte of SRAM and cache controller logic just to make decode tolerable at scale.