The Performance Monitoring Counter Overflow Interrupt: How Profilers Sample Without Polling

2026-06-11

When you run perf record -e cycles ./myprogram, the profiler isn't polling the CPU in a loop asking "what are you doing now?" That would destroy the workload and give you garbage data. Instead, it's using a hardware mechanism called PMI — the Performance Monitoring Interrupt — to make the CPU interrupt itself every N events.

Here's how it works. The PMU has counters (typically 4 general-purpose, 3 fixed on Intel). You program one to count cycles, but instead of starting at zero, you start it at 2^48 - sample_period. The counter increments on each cycle. When it overflows (wraps to zero), the local APIC delivers a Performance Monitoring Interrupt — vector 0xFE on most kernels — and the handler captures RIP, the call stack, and PMU state into a ring buffer that perf reads via mmap.

The killer detail: the interrupt is not precise on older CPUs. Out-of-order execution means by the time the interrupt fires, dozens of instructions past the "guilty" one have retired. Your profile blames the wrong line. This is why Intel added PEBS (Precise Event-Based Sampling) — when a counter overflows, the CPU writes a snapshot of architectural state to a memory buffer at instruction retirement, so RIP points to the actual instruction that caused the event. AMD's equivalent is IBS.

Concrete example: You profile a hot loop with perf record -e cache-misses ./app. Without PEBS, samples are attributed to instructions 10-50 bytes downstream of the actual load that missed. You'll spend an hour optimizing an ADD that wasn't the problem. Adding :pp (perf record -e cache-misses:pp) requests PEBS — now the sample points at the exact MOV that touched the cold cache line.

Rule of thumb: sample period directly trades accuracy for overhead. At 1000 Hz (1 sample/ms) overhead is ~1-2%. At 10,000 Hz it's 10%+, and the PMI itself starts perturbing the cache state you're trying to measure — you're measuring the profiler. Default perf targets 4000 Hz and auto-adjusts the period to hit that rate.

Why this matters operationally: the PMI runs in NMI context (Non-Maskable Interrupt) on Linux so it can profile code that disabled interrupts — including spinlocks and kernel critical sections. That's why perf can show you "this kernel spinlock is hot" but it also means a buggy PMI handler can deadlock the whole machine. The kernel walks the stack from NMI context using only frame pointers or DWARF unwind info pre-loaded into the perf ring buffer.

Key Takeaway: Profilers don't poll — they program a hardware counter to overflow every N events, triggering an NMI that captures the instruction pointer, with PEBS providing the precise RIP that out-of-order execution would otherwise smear.

All newsletters