2026-07-08
You've profiled code with perf record a thousand times. But perf itself is just a userspace program calling one syscall: perf_event_open(2). Understanding it explains why sampling at 10,000 Hz doesn't destroy your program's performance — and why it sometimes does.
The syscall. perf_event_open returns a file descriptor tied to a hardware or software event (cycles, cache misses, page faults, context switches). You configure it with a struct perf_event_attr: which event, whether to sample or just count, sample period, what to record on each sample (registers, callchain, timestamps, branch stack).
The problem it solves. If every sample required a syscall to read, sampling at 10 kHz would mean 10,000 read() calls per second per counter — thousands of context switches just to profile. Instead, the kernel writes samples into an mmap'd ring buffer that userspace reads directly.
The ring buffer layout. After perf_event_open, you mmap the fd with a size of (1 + 2^n) * PAGE_SIZE. The first page is the metadata header (head/tail pointers, current counter value); the remaining 2^n pages are the data ring. The kernel writes; userspace reads. No syscall on the fast path — just memory reads and a compiler_barrier() after updating the tail pointer.
Concrete example. perf record -F 10000 -e cycles ./myprog. Under the hood: one perf_event_open per CPU (so 16 fds on a 16-core box), each mmap'd with 128 pages (default --mmap-pages 128). At 10 kHz across 16 cores that's 160,000 samples/sec written directly to userspace-visible memory. perf polls with poll(2) only when the buffer fills — wakeup_events or wakeup_watermark in the attr controls how often the kernel signals.
The sizing rule of thumb. Each sample averages ~64 bytes (more with callchains — 8 bytes per frame). At 10,000 samples/sec on a busy core with 32-frame stacks, that's ~3.2 MB/sec. With 128 pages (512 KB), the buffer fills in ~150 ms — plenty of headroom for perf's userspace to drain it. Undersize it and you get the dreaded [LOST] records; oversize it and you waste locked memory (counts against RLIMIT_MEMLOCK).
The subtle gotcha. The counter value in the metadata page is updated via rdpmc semantics — you can read it from userspace with the PERF_EVENT_IOC_REFRESH pattern using a seqlock-style version field (lock). Read lock, read counter, read lock again — retry on mismatch. That's how libraries like PAPI read hardware counters in ~10 ns without a syscall.
perf_event_open plus an mmap'd ring buffer turns profiling from "one syscall per sample" into "one syscall per buffer full," which is why sampling at 10 kHz costs percent, not multiples.
