2026-06-19
You allocated a 1GB buffer with malloc(). It came back as 262,144 separate 4KB pages. A few minutes later, without you doing anything, those pages have silently collapsed into 512 huge 2MB pages — your TLB miss rate dropped 50x and you never called madvise(). That's khugepaged, a kernel thread that prowls process address spaces looking for opportunities to promote.
The mechanism is surprisingly aggressive. khugepaged wakes up every scan_sleep_millisecs (default 10s) and scans pages_to_scan PTEs per pass (default 4096). For each 2MB-aligned virtual region in a VMA flagged VM_HUGEPAGE (or all VMAs, under always mode), it checks: are at least max_ptes_none of the 512 PTEs actually populated? If yes, it allocates a fresh 2MB page from the buddy allocator, copies the contents of all populated 4KB pages into it, atomically swaps the PMD entry from "points to PTE table" to "points to 2MB page," and frees the old 4KB pages back to the buddy allocator.
Why this is harder than it sounds: the swap has to happen under mmap_sem write-locked plus the PMD lock, with every PTE in the range checked for races against page faults, COW, and migration. If any thread takes a fault on a page being collapsed, the operation aborts. On heavily multi-threaded processes, khugepaged can churn for hours making no progress.
Real-world example: Redis on a 512GB instance with anti-fragmentation off would routinely show 30-50ms latency spikes during background save (BGSAVE). The fork() child inherits a COW snapshot — and khugepaged, seeing the now-mostly-untouched parent pages, attempts to collapse them. Each collapse triggers a CPU stall on every core that holds a TLB entry for the affected PMD (a TLB shootdown). Antirez disabled THP for Redis in 2014 by writing never to /sys/kernel/mm/transparent_hugepage/enabled; the recommendation persists today for any fork-heavy workload.
Rule of thumb: THP wins for long-lived, large, sequentially-accessed allocations (databases, ML model weights, JVM heaps with G1GC). It loses for fork-heavy workloads, sparse memory access patterns, and anything sensitive to tail latency. The math: each 2MB collapse copies 2MB of memory and shoots down TLBs on ~N cores. If your process has 64 cores and the page wasn't going to be reused much, you just spent ~10μs of wall time and ~640μs of CPU time to save maybe 511 future TLB misses at ~100ns each = 51μs. Break-even is poor for cold data.
Check it live: grep AnonHugePages /proc/$PID/status shows how many bytes of your process are currently backed by THP.
