2026-07-14
When pthread_cond_broadcast() wakes every thread waiting on a condition variable, a naive implementation would call FUTEX_WAKE on all of them. Every woken thread immediately tries to grab the associated mutex, but only one can win. The other 999 threads run, contend on the mutex futex, fail, and go back to sleep — a classic thundering herd. Wall clock burned, cache lines bounced, scheduler thrashed.
FUTEX_REQUEUE solves this. Instead of waking N waiters on futex A, it wakes one waiter on A and moves the remaining waiters to the wait queue of futex B, all inside the kernel, without any thread ever running. Those threads stay blocked, but now they're queued on the mutex futex instead of the condvar futex. When the winner eventually releases the mutex, the next waiter is already positioned to be woken normally.
The API: futex(uaddr, FUTEX_REQUEUE, wake_count, requeue_count, uaddr2). It wakes up to wake_count waiters on uaddr, then moves up to requeue_count more to uaddr2. glibc typically passes 1 and INT_MAX.
FUTEX_CMP_REQUEUE adds a safety check: it takes an extra expected_value and atomically verifies *uaddr == expected_value before doing anything. This closes a race where a waker between the userspace decision to broadcast and the kernel's requeue could have already changed the condvar's state (e.g., another thread called signal). Without the compare, you might requeue waiters that should have been left alone. Every modern glibc uses FUTEX_CMP_REQUEUE, never plain FUTEX_REQUEUE.
Concrete example: A worker pool has 1000 threads waiting on a condition variable for new work. A producer calls pthread_cond_broadcast(). Without requeue: 1000 futex wakeups, 1000 syscall returns, 999 mutex-lock failures, 999 futex sleeps — roughly 2000 context switches for one broadcast. With FUTEX_CMP_REQUEUE: 1 wakeup, 999 in-kernel queue transfers, ~2 context switches. Measured on Linux, a broadcast to 1000 waiters drops from ~40ms of scheduler work to under 100µs.
Rule of thumb: Each avoided spurious wakeup saves roughly one context switch pair (~2–5µs on modern hardware) plus L1/L2 cache pollution. For N waiters where only one can proceed, requeue saves approximately (N-1) × 4µs. At N=1000, that's 4ms of pure overhead eliminated — per broadcast.
The requeue mechanism is why pthread_cond_broadcast scales at all. It's also why glibc's condvar implementation carefully maintains a separate "internal futex" address distinct from the mutex — the requeue destination must be stable across the operation.
