2026-06-09
On a server with 32MB of L3 cache shared across 16 cores, one cache-thrashing workload can evict every line another workload depends on. The victim suddenly sees its working set fall out of LLC, its memory bandwidth demand spikes, and its tail latency doubles — even though nothing about its code changed. This is the noisy neighbor problem, and prior to ~2015 the only fix was physical separation.
Intel's Cache Allocation Technology (CAT), part of Resource Director Technology (RDT), lets you carve the L3 into named partitions called Class of Service (CLOS) IDs. Each CLOS holds a bitmask — the Capacity Bitmask (CBM) — where each bit represents one "way" of the LLC's associativity. A 20-way LLC with CBM 0xFFC00 reserves the upper 10 ways; CBM 0x003FF reserves the lower 10. Cache lines fetched while a core runs under that CLOS can only be installed in the allowed ways.
Configuration happens through MSRs: IA32_L3_QOS_MASK_n (0xC90+n) defines the bitmask for CLOS n, and IA32_PQR_ASSOC (0xC8F) selects which CLOS the current core uses. Linux exposes this via resctrl, a pseudo-filesystem at /sys/fs/resctrl/:
mkdir /sys/fs/resctrl/database — creates CLOSecho "L3:0=0xff000;1=0xff000" > schemata — give it the upper 8 ways on both socketsecho $PID > tasks — assign a processReal-world example: Google's Borg and Meta's container runtimes use CAT to reserve LLC for latency-sensitive services (search, ads ranking) while batch jobs share a smaller partition. A measured case: a Redis instance co-located with a Spark shuffle saw p99 latency drop from 14ms to 2.1ms after pinning Redis to a 75%-LLC CLOS and Spark to the remaining 25%.
Rule of thumb for sizing partitions: measure the working set with perf stat -e LLC-loads,LLC-load-misses. If your hot path's resident footprint is W megabytes on an N-megabyte LLC with K ways, give it at least ceil(W/N * K) + 2 ways. The +2 absorbs conflict misses from set-associative hashing. Going below the working set is catastrophic; going above wastes shared capacity.
Important gotchas: CBMs must be contiguous bit-runs on most CPUs — 0b10110 faults. Partitions may overlap, which is how you do soft priorities (shared region + exclusive region). And CAT controls allocation, not access: a CLOS-0 core can still hit on a line installed by CLOS-1 — partitioning only prevents new installs, so the effect builds gradually as old lines age out.
