Direct-mapped caches are fast and cheap, but they have a brutal failure mode: conflict misses. Two addresses that map to the same set evict each other on every access, even when the rest of the cache is empty. Set-associative caches fix this by adding ways, but every way you add costs power, area, and tag-comparison time on the hot path. Norman Jouppi proposed a sneakier fix in 1990: keep a tiny, fully-associative victim cache next to the main cache that catches every line the main cache evicts.

The structure is small — typically 4 to 16 entries — and sits between the L1 and the next level of memory. When L1 evicts a line, the line doesn't go straight to L2; it goes into the victim cache. On the next L1 lookup:

• L1 and the victim cache are probed in parallel.
• If L1 hits, normal fast path — victim cache result is discarded.
• If L1 misses but the victim cache hits, the two lines swap: the victim-cache line moves into L1, and the line L1 was about to evict goes into the victim cache.
• If both miss, fetch from L2 and evict normally.

The swap is the clever part. It costs one extra cycle but turns what would have been an L2 miss (often 10–30 cycles) into a near-hit. Because the victim cache is fully associative over a handful of entries, it tolerates any conflict pattern the main cache can't.

Concrete example: A direct-mapped 8KB L1 with 32B lines has 256 sets. Two hot variables at addresses 0x1000 and 0x3000 both hash to set 0 and thrash. Without a victim cache, every access to one evicts the other — 100% miss rate on those two lines. Add a 4-entry victim cache: the evicted line lands one cycle away, and the next access swaps it back. The thrash becomes a 1-cycle penalty instead of a 20-cycle L2 round trip.

Rule of thumb: Jouppi's original paper showed a 4-entry victim cache removes roughly 25–50% of conflict misses for a direct-mapped L1 — performance close to 2-way set-associative at a fraction of the area. Diminishing returns kick in fast past 8 entries because hot conflict sets are usually small.

Modern CPUs rarely ship explicit victim caches at L1 anymore (associativity got cheap), but the idea lives on: AMD's Zen architectures use the L3 as a victim cache for L2 — L3 lines only exist because L2 evicted them, which makes L3 enormous-but-exclusive instead of duplicating L2's contents. Same trick, different level of the hierarchy.