The RCU Grace Period Detector: How the Kernel Knows When It's Safe to Free Memory Nobody Is Reading

2026-06-29

RCU readers don't take locks — they just dereference pointers. So when a writer swaps a pointer and wants to free the old object, how does the kernel know every reader that might have grabbed the old pointer has finished with it? The answer is the grace period detector, and its core trick is one of the most elegant hacks in the kernel.

The fundamental observation: RCU readers are forbidden from blocking. An RCU read-side critical section is bounded by rcu_read_lock() and rcu_read_unlock(), which in the classic kernel implementation are literally preempt_disable() and preempt_enable() — zero instructions on a non-preemptible kernel. This means: if a CPU has performed a context switch, it cannot be inside an RCU read-side critical section that started before that switch.

The algorithm: To declare a grace period complete, the kernel waits until every CPU has passed through a quiescent state — a context switch, a trip to idle, or a return to user mode. Once each CPU has been observed in one of these states at least once after the grace period began, every pre-existing reader is guaranteed to have finished. The old memory can be freed.

Concrete example: Writer updates a network routing table at T=0 and calls call_rcu(old_entry, kfree). The grace period detector records: "we need all 16 CPUs to quiesce after T=0." CPU 3 is running a long syscall — the detector waits. At T=12ms, CPU 3 returns to userspace (quiescent state). All 16 CPUs have now quiesced. The callback fires at T=12ms and frees the old routing entry. No reader can possibly still hold a pointer to it.

Implementation detail: The detector uses a hierarchical tree of rcu_node structures to avoid cache-line contention. On a 128-core box, leaf nodes group 16 CPUs each; only when all 16 in a group have quiesced does that leaf update its parent. Without the tree, every CPU would slam the same atomic counter on every context switch.

Rule of thumb: A grace period typically takes 10–100 ms on a busy server — bounded below by the scheduler tick. This is why call_rcu() is asynchronous and why memory reclamation in RCU-heavy workloads can lag noticeably. If you ever see rcu_sched detected stalls in dmesg, it means some CPU went 21+ seconds without quiescing — usually a stuck kernel thread or a runaway interrupt handler.

Synchronous variant: synchronize_rcu() blocks the caller until a full grace period elapses. Never call it from an atomic context, and never from a path that holds a lock readers might wait on — instant deadlock.

Key Takeaway: RCU reclaims memory safely by waiting for every CPU to context-switch at least once — proving no reader from before the update can still be running.

All newsletters