2026-06-27
The buddy allocator carves memory into power-of-two blocks, but it doesn't operate on one flat pool of RAM. Linux partitions physical memory into zones, and a single allocation request walks a fallback list across them. Understanding zones explains why your 64GB server can fail a 1KB allocation while having 50GB free.
On x86-64, the three zones that matter are:
Each zone has its own buddy allocator, its own free lists, and its own watermarks (min, low, high). When you call kmalloc(size, GFP_KERNEL), the kernel picks a preferred zone from the GFP flags, then walks a zonelist in fallback order. GFP_DMA32 means "must come from ZONE_DMA32 or below" — it cannot fall back upward.
Real-world example: A NIC driver allocates DMA buffers with GFP_DMA32 because the card has a 32-bit DMA mask. On a 256GB server under load, ZONE_NORMAL has plenty of free pages, but ZONE_DMA32 (only ~4GB of address space, minus kernel structures already pinned there) gets fragmented. The driver's allocation fails with -ENOMEM even though free -m shows 200GB available. Fix: set the device's DMA mask to 64 bits if the IOMMU is enabled, or enable swiotlb bounce buffers.
The watermark rule of thumb: kswapd wakes when a zone drops below its low watermark and reclaims until it hits high. Direct reclaim (synchronous, in your allocation's call stack) kicks in below min. The default min is roughly sqrt(16 × zone_size_KB) KB — for a 4GB zone, around 8MB. Tune via /proc/sys/vm/min_free_kbytes; raise it on machines that do bursty allocations to keep kswapd ahead of demand.
Check zone state with /proc/zoneinfo or /proc/buddyinfo. The latter shows free blocks per order per zone — if order-0 in ZONE_DMA32 has 50,000 free pages but order-3 has zero, you have a fragmentation problem that a flat "free memory" number completely hides.