A 4-way set-associative cache normally reads all four ways in parallel on every access, then uses the tag comparators to pick the winning way's data. That's wasteful: three of the four SRAM reads are thrown away, burning dynamic power for nothing. On a hot L1 doing a billion accesses per second, that's a significant chunk of the core's energy budget.

Way-prediction fixes this. Instead of speculatively reading every way, the hardware predicts which way the line will be in, reads only that one way, and verifies the prediction with the tag check in parallel. If correct: one SRAM read, normal latency. If wrong: re-read the other ways next cycle and take a one-cycle penalty.

The predictor is usually tiny — a small table indexed by some bits of the PC or the virtual address, storing the way-number (just 2 bits for a 4-way cache) that hit last time at that index. On well-behaved code (loops hitting the same data repeatedly), prediction accuracy runs 85–95%.

Real-world example: the Alpha 21264 (1998) used way-prediction on its 64KB 2-way L1 data cache. It hit single-cycle latency that a fully parallel 2-way design couldn't match at the same frequency, because reading only one way let the SRAM bitlines stay short and the sense amp fire sooner. ARM's Cortex-A series and Apple's M-series cores still use variants of this trick on L1 to keep load-use latency at 3–4 cycles while shaving cache power.

Rule of thumb for energy savings:

• N-way cache, prediction accuracy p, mispredict re-reads all N ways
• Energy per access ≈ p × 1 way + (1−p) × (N+1) ways
• For 4-way at 90% accuracy: 0.9 × 1 + 0.1 × 5 = 1.4 way-reads per access, vs 4 for non-predicted. 65% energy reduction on the data array.

The catches are real. A misprediction stalls the load pipeline one cycle, which ripples into the scheduler if the load is on a critical path. Worse, way-prediction tables are side-channel leaky: the timing difference between hit-and-predicted-correctly vs hit-and-mispredicted is observable, and researchers have used this to leak secrets across security domains (the "Take A Way" attack on AMD's L1D, 2020).

Modern designs mitigate this by partitioning the predictor per-process or flushing it on context switch — trading a bit of accuracy for not leaking your AES key to a JavaScript timer.