The Last-Level Cache Slice Hashing: Why Your L3 Access Latency Isn't Uniform Across Cores

2026-09-13

Modern CPUs don't have a single monolithic L3 cache. The last-level cache is sliced — chopped into equal chunks and physically distributed, one slice per core, connected by the on-chip interconnect (ring on older Intel, mesh on Xeon SP, Infinity Fabric on AMD). Every core can reach every slice, but not at the same speed.

The question is: which slice holds a given cache line? If the CPU used the low bits of the address (like it does inside L1), every consecutive line would hit the same slice and one slice would melt. So the address gets fed through a hash function — typically an XOR tree over the high physical address bits — and the result picks the slice. This spreads lines pseudo-randomly across slices, giving every slice equal load on average.

The catch: the hash function is undocumented and different per SKU. Intel changes it between generations and even between core counts within the same generation (a 10-core Skylake-X uses a different hash than an 18-core one, because the slice count is different). Researchers have reverse-engineered these hashes for specific chips using the uncore performance counters.

Why you should care:

Rule of thumb: On a mesh-based Xeon, expect L3 hit latency to vary by ~1.5 cycles per mesh hop. A 6x6 mesh has a worst-case diameter of 10 hops, so slice placement can add 15+ cycles on top of the base ~40-cycle L3 hit.

Concrete example: On a 24-core Xeon Platinum 8260, a core in the top-left corner accessing a line whose hash lands in the bottom-right slice will see roughly ~55 cycles of L3 hit latency, versus ~40 cycles for a local slice. That 15-cycle penalty is invisible to profilers that only report "L3 hit rate" — you need uncore counters (UNC_CHA_TOR_INSERTS) to see it.

Key Takeaway: "L3 hit" isn't a single number — the slice hash scatters your data across the die, and how far your data lives from your core determines whether that hit costs 40 cycles or 55.

All newsletters