2026-07-23
You've already got SLRU with ghost entries. You've added frequency-weighted promotion. Now here's the failure mode nobody warns you about: a key gets hammered for ten seconds during a batch job, accumulates a huge frequency count, and then sits in your protected segment for hours doing nothing useful. Frequency without a time window is just lifetime popularity, and lifetime popularity is a terrible predictor of future access.
Promotion windowing fixes this by only counting accesses that happened inside a sliding time window (say, the last 5 minutes) when deciding whether a probationary key deserves promotion to the protected segment. Old hits decay out; recent bursts count. The key still lives in the cache — you're only changing the promotion decision, not eviction.
How it works:
Real-world example: A news site's article cache. When a story trends, it gets 50,000 hits in an hour and its lifetime frequency shoots up. Without windowing, that article sits in the protected segment for the next 48 hours while newer stories fight for probationary slots. With a 30-minute promotion window, once the trend dies, the article stops getting promoted on subsequent accesses and naturally drifts out — freeing protected space for the next trending story.
Rule of thumb: Set the window to roughly 2–3× your typical hot-key lifespan. For CDN edge caches serving trending content, 15–30 minutes works well. For database buffer pools where working sets shift by workload phase, match the window to your phase duration (often 5–15 minutes). Too short and you thrash — legitimately hot keys keep failing promotion. Too long and you're back to lifetime frequency.
Implementation cost: A 4-slot timestamp ring per key adds ~32 bytes. On a 10M-entry cache, that's 320MB — non-trivial. Cheaper alternative: an exponentially-decayed counter (8 bytes) with decay applied lazily on access. You lose precision but keep the "recent bursts matter more" property.
When to skip it: If your access patterns are stationary (same keys hot for weeks), windowing adds cost without benefit. It shines when workloads have temporal locality of popularity — trending content, time-of-day patterns, campaign-driven traffic.
