The Segmented LRU with Multi-Tier Ghost Entries: When One Ghost List Per Segment Isn't Enough

2026-07-22

You've segmented your LRU into probationary and protected tiers. You've added ghost entries so evictions leave a trace. But you're still missing hits — items evicted from the probationary ghost list return later and you've forgotten them. Enter multi-tier ghost entries: layered eviction memory that remembers where in the hierarchy an item died, not just that it did.

The structure has four regions instead of two:

On a miss, you check both ghost tiers. A hit in recent ghosts promotes the incoming item straight to probationary with a modest priority. A hit in frequent ghosts promotes it to protected immediately — the ghost tier remembers this key was previously hot, so we skip probation. The cost per key in ghost lists is ~16 bytes (hash + tier flag + timestamp), so 100K ghost entries = 1.6MB — trivial compared to the value payloads you're protecting.

Real-world example: A CDN edge node caching image variants for an e-commerce site. Product hero images stay hot for months. But when a flash sale hits, thousands of newly-viewed thumbnails flood in and evict the heroes from protected → probationary → gone. With a single ghost list, when normal traffic resumes and someone requests a hero, you fetch from origin (200ms) and start probation from scratch. With frequent ghosts, that first post-sale request restores the hero directly to protected. One team reported a 12% hit rate lift during traffic spike recoveries with essentially zero extra RAM.

Rule of thumb: Size each ghost tier to roughly match its corresponding real tier. If protected holds 8000 entries and probationary holds 2000, give frequent ghosts ~8000 slots and recent ghosts ~2000. Total metadata overhead stays under 5% of total cache memory while doubling your effective "memory of what mattered."

Watch out for: ghost lookups on every miss add latency. Use a shared hash index across both ghost tiers with a tier-flag byte rather than two separate maps — one lookup, not two. Also, don't let ghost entries outlive their usefulness: age them out on a coarse timer (say, 10× the average protected residency time) so a key that was hot last Tuesday doesn't unfairly leapfrog a genuinely trending item today.

Key Takeaway: Multi-tier ghost lists let your cache remember not just that an item was evicted, but how important it was — so returning items get restored to the tier they earned, not the bottom of the pile.

All newsletters