2026-07-22
You've segmented your LRU into probationary and protected tiers. You've added ghost entries so evictions leave a trace. But you're still missing hits — items evicted from the probationary ghost list return later and you've forgotten them. Enter multi-tier ghost entries: layered eviction memory that remembers where in the hierarchy an item died, not just that it did.
The structure has four regions instead of two:
On a miss, you check both ghost tiers. A hit in recent ghosts promotes the incoming item straight to probationary with a modest priority. A hit in frequent ghosts promotes it to protected immediately — the ghost tier remembers this key was previously hot, so we skip probation. The cost per key in ghost lists is ~16 bytes (hash + tier flag + timestamp), so 100K ghost entries = 1.6MB — trivial compared to the value payloads you're protecting.
Real-world example: A CDN edge node caching image variants for an e-commerce site. Product hero images stay hot for months. But when a flash sale hits, thousands of newly-viewed thumbnails flood in and evict the heroes from protected → probationary → gone. With a single ghost list, when normal traffic resumes and someone requests a hero, you fetch from origin (200ms) and start probation from scratch. With frequent ghosts, that first post-sale request restores the hero directly to protected. One team reported a 12% hit rate lift during traffic spike recoveries with essentially zero extra RAM.
Rule of thumb: Size each ghost tier to roughly match its corresponding real tier. If protected holds 8000 entries and probationary holds 2000, give frequent ghosts ~8000 slots and recent ghosts ~2000. Total metadata overhead stays under 5% of total cache memory while doubling your effective "memory of what mattered."
Watch out for: ghost lookups on every miss add latency. Use a shared hash index across both ghost tiers with a tier-flag byte rather than two separate maps — one lookup, not two. Also, don't let ghost entries outlive their usefulness: age them out on a coarse timer (say, 10× the average protected residency time) so a key that was hot last Tuesday doesn't unfairly leapfrog a genuinely trending item today.
