Hardware prefetchers don't just predict — they self-grade. Every prefetcher in a modern CPU carries a small saturating counter (typically 2-4 bits) tracking how often its recent guesses actually got consumed by a real demand load. This is the confidence counter, and it's the throttle that keeps prefetching from destroying the cache it's trying to help.

The mechanism is simple. When the prefetcher issues a line and a demand load later hits it before eviction: counter++. When a prefetched line gets evicted unused: counter--. The current counter value determines prefetch distance (how far ahead to fetch) and prefetch degree (how many lines per trigger). High confidence: fetch 8 lines ahead, 4 at a time. Low confidence: fetch 1 line, or stop entirely.

Why this matters: an unthrottled prefetcher on the wrong workload is catastrophic. It pollutes L2 with useless lines, steals memory bandwidth from demand loads, and triggers needless coherence traffic. Intel's L2 streamer and AMD's Zen prefetchers both implement per-stream confidence with explicit "useful prefetch" feedback wires from the LLC.

Concrete example: a hash table probe sequence. The first 100 inserts look random to the stride prefetcher — confidence drops to 0, prefetcher goes dormant. Then your code transitions to scanning the table's underlying array for resizing. The prefetcher sees stride=64, but its counter is still pinned low from the random phase. For the first ~64 cache lines of the scan, you get zero prefetching — every load is a cold miss to DRAM (~80 ns each). Only after the confidence counter climbs back up does throughput recover. This is why benchmarks often show a "warmup tail" of 5-10 µs at workload transitions that has nothing to do with cache fill.

Rule of thumb: a 2-bit saturating counter needs roughly 4 consecutive correct predictions to go from cold (00) to maximum aggression (11), and 4 wrong ones to fully shut down. At ~10 ns per L2 access during training, that's ~40 ns per phase transition — but only if your access pattern is steady. Mixed patterns can leave the counter oscillating in the middle, giving you mediocre prefetching forever.

The practical consequence: workload phase changes are expensive in ways profilers rarely show. The cache is warm, the TLB is warm, but the prefetcher's opinion of you is cold.