2026-09-13
You've used __atomic_fetch_add() or std::atomic::fetch_add() a hundred times. On x86, that intrinsic compiles down to a single instruction you've probably never written by hand: XADD (exchange-and-add), prefixed with LOCK when you need atomicity across cores.
The semantics are subtle. XADD takes a register and a memory location. In one atomic step it does: temp = mem; mem = mem + reg; reg = temp. The register receives the old value of memory, and memory receives the sum. That's fetch-and-add — the fundamental primitive behind reference counters, sequence number generators, and ticket spinlocks.
Why is this a distinct instruction? Because plain LOCK ADD throws away the old value. If you want to know "what was the counter before I incremented it?" — say, to hand out unique IDs — ADD forces a compare-and-swap loop. XADD gets you the answer in one atomic operation, no retry.
Concrete example — a ticket lock:
next_ticket and now_serving.my_ticket = XADD(next_ticket, 1). You atomically grab the next number and increment the counter for whoever comes after you.now_serving == my_ticket.now_serving++ (no atomic needed — you hold the lock).This is a fair FIFO lock in ~5 instructions. Linux used exactly this scheme from 2008 until qspinlocks replaced it around 2014 (ticket locks scaled badly past 8 cores because every release invalidated everyone's cache line).
Cost calculation: An uncontended LOCK XADD on modern Intel is roughly 15–25 cycles if the line is already in the local L1 in M state. If another core owns it, expect 60–120 cycles for the cache line to migrate via the ring bus. Rule of thumb: an atomic op on a contended line costs about the same as an L3 hit — around 40ns. A million atomic increments on one shared counter from 16 cores? ~40ms of wall-clock time just for coherence traffic.
Gotcha: XADD without the LOCK prefix is not atomic across cores — it's only atomic against interrupts on the same core. GCC's __atomic_fetch_add with __ATOMIC_RELAXED still emits LOCK XADD on x86 because the ISA requires the prefix for cross-core atomicity; there is no "cheap atomic" on x86. If you see XADD without LOCK in disassembly, someone is doing per-CPU accounting (like this_cpu_add_return in the kernel), not sharing across cores.
