2026-07-11
Classical LRU caches have a known weakness: a single burst of one-shot reads (a scan, a crawler, a report) evicts your hot working set. Pure LFU has the opposite problem: it clings to items that were popular last week but are dead today. W-TinyLFU, the admission policy behind Caffeine (Java) and used by Ristretto (Go) and parts of Redis, solves both by splitting the cache into two regions and gating promotions with a frequency sketch.
The structure has three parts:
When an item is evicted from the window, it doesn't die immediately. It challenges the LRU victim of the main cache's probation segment. TinyLFU asks: "Which of these two has been accessed more often recently?" The winner stays; the loser is evicted. This is the admission filter — the main cache never accepts a newcomer unless it has proven itself more valuable than what it would displace.
Real example: A CDN edge serving mixed traffic — 95% requests hit a hot set of ~10K URLs, plus a background crawler sweeping millions of URLs once each. Under LRU, the crawler's scan evicts the hot set every few minutes, tanking hit rate to ~40%. Under W-TinyLFU, crawler URLs enter the window, get evicted quickly, and fail the frequency contest against the hot URLs in probation. Hit rate stays near 92%. Caffeine benchmarks routinely show W-TinyLFU beating LRU by 10–40 percentage points on scan-heavy workloads, with near-optimal (Belady) performance on Zipf distributions.
Rule of thumb: Size the window at 1% of the cache for standard workloads. If your workload is recency-heavy (news feeds, chat), Caffeine's adaptive variant will hill-climb the window ratio up to ~20% automatically. The frequency sketch needs roughly 8× cache-size counters at 4 bits each — so a 1M-entry cache costs ~4 MB of sketch overhead, which is negligible.
The trap to avoid: don't reach for W-TinyLFU when a plain LRU is fine. If your working set fits in cache, LRU is simpler and equivalent. W-TinyLFU pays off when your workload has a long tail plus scans — that's when the admission filter earns its keep.
