The 2Q Cache with Ghost Entries: Remembering Evictions to Fight Cold Starts

2026-07-20

Standard 2Q uses three queues: A1in (FIFO for newcomers), Am (LRU for proven hot items), and A1out (a ghost queue of keys recently evicted from A1in). The ghost queue is the clever bit — it holds keys only, no values. When a key reappears and its ghost is still in A1out, 2Q promotes it directly to Am. Without the ghost, that key would start over in A1in and get evicted again on the next scan.

The problem 2Q's ghosts solve is the cold-start penalty for genuinely hot items. Imagine a product page that gets hit every 30 seconds, but a batch job periodically floods the cache with 100,000 one-off keys. Each flood pushes the product page out of A1in before it's touched twice. Without ghosts, the product page keeps re-entering as a "newcomer" and getting evicted again — permanent second-class citizenship. With ghosts, the second miss lands in A1out, so the third access sends it straight to Am where scan traffic can't reach it.

The sizing rule of thumb: A1in ≈ 25% of cache capacity, Am ≈ 75%, and A1out (ghost) ≈ the size of Am measured in keys. Since ghost entries store only a key hash (~16 bytes) instead of a value (often KB), the ghost queue costs roughly 1-2% of total cache memory while doubling the effective "memory" of what's been seen recently.

Concrete example: A CDN edge cache holds 10GB of video segments (avg 2MB each = ~5,000 entries). Am holds ~3,750 segments, A1in holds ~1,250, and A1out ghosts ~3,750 keys × 16 bytes = 60KB. During a viral event where a 5-minute clip gets requested 100 times/second interleaved with thousands of cold requests, the clip's segments hit the ghost list on their second request and get pinned in Am — surviving the noise. Measured hit rate improvement over plain 2Q without ghosts: typically 8-15% on scan-heavy workloads.

Common mistakes: undersizing A1out (ghosts expire before the reference pattern repeats), storing full keys instead of hashes (blowing the memory budget), and forgetting to remove a key from A1out when it's promoted to Am (leaking ghost slots). Also: don't confuse A1out with an admission filter like TinyLFU — ghosts remember individual evictions, while TinyLFU tracks aggregate frequency. They compose well; many production caches use both.

See it in action: Check out (18+)🃏🎭May I Steal Your Heart? Take Your Time Deciding~🎭🃏 by TauTassieTiger to see this theory applied.
Key Takeaway: 2Q's ghost queue is a cheap memory trick that turns "evicted once" into "seen before" — protecting hot items from scan traffic at ~1% of the cache's memory cost.

All newsletters