The Segmented LRU with Ghost Entries: Combining SLRU's Structure with 2Q's Memory

2026-07-21

Plain SLRU splits the cache into a probationary segment and a protected segment. New items land in probation; a second hit promotes them to protected. When protected fills, the least-recently-used protected item gets demoted back to probation. This shields hot items from one-hit-wonder scans.

But plain SLRU has a blind spot: once an item is evicted from probation, it's forgotten. If that item comes back a minute later, it starts fresh in probation and gets no credit for its prior existence. Scan-heavy workloads keep re-evicting the same warm items over and over.

SLRU with ghost entries fixes this by keeping a third list — a ghost list of keys (no values) recently evicted from probation. On a miss, before inserting into probation, you check the ghost list. If the key is there, you know it's a returning item that got squeezed out, so you insert it directly into the protected segment. The ghost list turns "re-appeared quickly" into a promotion signal without needing two actual hits.

Concrete example — image thumbnail cache at a photo-sharing site:

When a nightly re-indexer scans 500,000 old thumbnails, they flood probation and evict warm items. Under plain SLRU, when a real user requests one of those warm items 30 seconds later, it goes through probation again and may get evicted a second time by the same scan. Under SLRU-with-ghosts, the ghost hit sends it straight to protected — safe from the scan.

Rule of thumb for sizing: ghost list should hold at least as many keys as the probationary segment holds values. If probation caches 20,000 items, keep ghost entries for the last 20,000+ evictions. Beyond that, diminishing returns — ghost lookups more than one probation-lifetime old rarely help.

Watch out for:

Key Takeaway: Adding a ghost list of evicted keys to SLRU lets returning items skip probation and land directly in protected, defending hot data against scans that would otherwise evict it twice.

All newsletters