2026-07-16
LRU evicts what nobody touched lately. LFU evicts what nobody asks for. Both are wrong half the time. LRU dies under scan workloads (one pass through cold data flushes your hot set). LFU dies under shifting workloads (yesterday's hot key stays cached forever because its counter is high). ARC (Adaptive Replacement Cache), invented by IBM in 2003, sidesteps the argument by running both policies at once and letting the workload vote.
ARC splits the cache of size c into four LRU lists:
T1 + T2 hold real cached data, capped at c. B1 + B2 hold just the keys of what was recently kicked out. A tunable parameter p decides how much of c belongs to T1 versus T2.
The magic is in p. On a hit in B1 (a ghost of something recently evicted for being "seen once"), ARC concludes: "we're throwing out recency data too aggressively" and grows p, giving T1 more room. On a hit in B2, ARC concludes: "we're throwing out frequency data too aggressively" and shrinks p, giving T2 more room. The ghosts are how the cache learns from its own mistakes.
Real-world example: PostgreSQL used ARC briefly in 2003 before backing it out (patent concerns — IBM held one until 2020). ZFS and OpenZFS use ARC as the primary read cache in their block layer. On a database workload that alternates between OLTP (frequency-heavy: same hot rows) and nightly reports (recency-heavy: scanning cold tables), pure LRU sees hit rates crater during the scan, while ARC's p shifts toward T2 during OLTP and back toward T1 as scan-like access appears — no manual tuning.
Rule of thumb: ARC typically delivers 5–15% higher hit rates than LRU on mixed workloads, at the cost of ~2× the metadata per entry (four list pointers plus ghost tracking). Ghost lists together consume about the same memory as the real cache in keys alone — budget for it. If your workload is stable (either purely recent or purely frequent), a simpler policy wins on overhead. ARC pays for itself when the access pattern changes.
Don't build ARC yourself. It's in caffeine (Java), lru_cache_2q variants, and every serious storage engine. Know it exists so you can reach for it when LRU vs LFU keeps flip-flopping in benchmarks.
