Every time your fast-path reader executes a memory barrier, you pay ~20-50 cycles for a fence that almost never matters — it's only there so the rare writer can synchronize with you. The membarrier() syscall inverts this cost: readers execute zero barriers, and the writer pays a much bigger cost (a syscall + IPI to every core) to force all other cores to barrier at once.

The magic command is MEMBARRIER_CMD_PRIVATE_EXPEDITED. When the writer calls it, the kernel sends an IPI to every CPU currently running a thread of your process. Each targeted core executes a full memory barrier before returning from the IPI. When the syscall returns, you have a guarantee: every reader thread has, at some point after the writer's store, executed a full barrier — so any subsequent load it does will see the writer's store.

Real-world example: LTTng's userspace tracer. Trace-point sites are hot — millions of hits per second across cores. Each site checks an "enabled" flag with a plain load. When you flip the flag off and want to free the ring buffer, you need every reader to have observed the disable before you free. Without membarrier(), each read would need smp_rmb() to pair with your writer's smp_wmb() — burning 20-40 cycles on every trace point when tracing is off. With membarrier(), the reader is a naked load; the disabler calls membarrier(MEMBARRIER_CMD_PRIVATE_EXPEDITED) once, then safely frees.

The math: Say you have a hot path hit 10^9 times/sec across 16 cores, with a config update once per minute. Symmetric barriers cost roughly:

• Traditional: 10^9 × 30 cycles = 3×10^10 cycles/sec — burned continuously
• membarrier(): ~5 μs per update × 1/60 sec = negligible; readers pay 0

Rule of thumb: use membarrier() when the read:write ratio exceeds ~1000:1 and readers are on a latency-sensitive path.

Variants worth knowing:

• MEMBARRIER_CMD_PRIVATE_EXPEDITED_SYNC_CORE — also serializes the instruction stream (needed for JITs patching code)
• MEMBARRIER_CMD_PRIVATE_EXPEDITED_RSEQ — pairs with rseq() critical sections
• MEMBARRIER_CMD_GLOBAL — non-expedited; waits for a scheduler tick rather than IPIing (cheaper for the caller, slower to complete)

Registration required: before using expedited variants, you must call membarrier(MEMBARRIER_CMD_REGISTER_PRIVATE_EXPEDITED). This tells the kernel your process opted in, so it knows which IPIs to send when other processes call it. Skipping registration returns EPERM.