A per-CPU variable is a single symbol in your code that expands to N independent copies at runtime — one per logical CPU. The kernel uses these everywhere it needs cheap statistics or lock-free bookkeeping: network packet counters, timer tick counts, slab allocator freelists, RCU state. The trick is that each CPU only ever touches its own copy, so there's no cache-line bouncing and no atomics needed for updates.

The layout is elegant. At boot, the kernel allocates one contiguous chunk of memory per CPU (the "percpu chunk"), and every per-CPU variable is placed at the same offset within each chunk. To find "my" copy, the CPU adds its per-CPU base pointer (stored in GS on x86-64, or TPIDR_EL1 on ARM64) to the variable's offset. That's a single instruction: %gs:offset.

The two flavors:

• Static per-CPU (DEFINE_PER_CPU(int, my_counter)): allocated at link time in a special .data..percpu section. The section is copied N times at boot.
• Dynamic per-CPU (alloc_percpu(struct foo)): carves from a runtime-managed chunk. Uses a first-fit allocator with populated/depopulated page tracking so unused pages can be returned to the buddy allocator.

Access rules: because the base pointer is per-CPU, you must not be preempted between reading the base and doing the access. Kernel code uses this_cpu_inc() / this_cpu_read() — these compile to a single instruction on x86 (incq %gs:offset) which is atomic against interrupts on that CPU by virtue of being one instruction. If you need multiple accesses, wrap them in get_cpu()/put_cpu(), which disables preemption.

Concrete example: network RX packet counters. On a 64-core box handling 10M pps, incrementing a single global counter with atomic_inc() would cost ~30 ns of cache-line bouncing per packet — 300 ms/sec of pure overhead. Each CPU incrementing its own per-CPU counter costs ~1 ns and never leaves L1. To read the total, iterate all CPUs and sum: for_each_possible_cpu(cpu) sum += *per_cpu_ptr(&counter, cpu). The read is racy but usually fine for statistics.

Rule of thumb: if a variable is written more than once per microsecond and read rarely (or approximately), make it per-CPU. If it's written rarely and read often, keep it shared. The cost of the summation loop scales with core count, so a 256-core sum is ~256 cache-line loads — still cheap for a debugfs read, but not for a hot path.

You can inspect the layout at /proc/vmallocinfo (look for "percpu") and see the chunks in /sys/kernel/debug/percpu_stats.