Hardware prefetchers issue speculative memory requests on top of the demand requests your code actually makes. When DRAM bandwidth is abundant, this is free performance. When the memory bus is already saturated, every prefetch request directly competes with demand loads for the same finite bandwidth — and since prefetches are speculative, you're paying real bandwidth for guesses.

The memory controller doesn't care that a request is "just a prefetch." It sees a queue of read requests and services them based on bank availability, row buffer state, and arbitration policy. A prefetch that arrives one cycle before a demand load will be serviced first, pushing the load behind it. The load now waits not just for DRAM, but for the prefetcher's guesses to finish.

The throttling mechanism: Modern prefetchers (Intel's L2 streamer, AMD's Region Prefetcher) monitor the memory controller's queue depth and outstanding miss count. When the MSHR pool fills past a threshold (~75% utilization on Skylake), the prefetcher backs off — issuing fewer requests or pausing entirely. AMD's Zen 4 will even kill in-flight prefetches that haven't been consumed by demand.

Real example: STREAM triad benchmark on a Xeon Platinum 8380 hits ~280 GB/s of theoretical DRAM bandwidth. With aggressive prefetching enabled in BIOS, you measure ~265 GB/s. Disable the L2 streamer and adjacent-line prefetcher — bandwidth jumps to ~278 GB/s. The prefetcher was guessing correctly, but its requests were redundant with demand loads on a perfectly-strided workload, and the speculation overhead cost ~5% of bus capacity. This is why HPC sites running memory-bound codes routinely disable hardware prefetchers and rely on software prefetch hints instead.

The asymmetry that bites you: A useful prefetch hides ~200 cycles of DRAM latency. A wasted prefetch on a saturated bus delays a demand load by ~50 cycles. So a prefetcher needs roughly 1 useful prefetch per 4 wasted prefetches just to break even — and accuracy collapses when multiple cores prefetch simultaneously, because each core's prefetcher trains in isolation.

Rule of thumb: If your workload's measured bandwidth is within 10% of theoretical peak, prefetching is now net-negative. Check via perf stat -e LLC-load-misses,offcore_requests.demand_data_rd: if the ratio of prefetched lines to demand misses exceeds 2:1 and bandwidth is saturated, disable the streamer.

The deeper lesson: prefetchers optimize for latency hiding, not bandwidth efficiency. When you cross from latency-bound to bandwidth-bound, the prefetcher's whole optimization target inverts — and the hardware doesn't always notice in time.