Both madvise(addr, len, MADV_DONTNEED) and madvise(addr, len, MADV_FREE) tell the kernel "I don't need these pages anymore." They look interchangeable. They aren't — and confusing them explains most "why doesn't my allocator return memory to the OS?" bug reports.

MADV_DONTNEED (eager): The kernel immediately unmaps the physical pages, zeros the PTEs, and drops RSS now. Your next access re-faults and gets a fresh zero-filled page (for anonymous memory). It's synchronous — you pay TLB shootdowns and page-table walk cost right away, but ps and /proc/self/status will show the memory gone the instant the syscall returns.

MADV_FREE (lazy, since Linux 4.5): The kernel just marks the pages as "reclaimable" via a per-VMA flag and sets PG_lru's swapbacked bit. The physical pages stay mapped. RSS does not drop. If you write to the page before memory pressure hits, the kernel clears the mark and you keep the original contents. If the reclaimer runs first, your next read returns zero. It's cheap (no TLB shootdown, no PTE clearing) but essentially invisible to userspace accounting.

The real-world trap: jemalloc switched from MADV_DONTNEED to MADV_FREE on Linux in 2016 (issue #387) to reduce page-fault overhead in busy servers. Users then filed hundreds of "memory leak" reports because RSS stayed flat under load. Jemalloc added an opt.muzzy_decay_ms knob and eventually a fallback path that issues MADV_DONTNEED after a delay. Go's runtime hit the same trap: since Go 1.12 it uses MADV_FREE, and Go 1.16 added GODEBUG=madvdontneed=1 so container users whose OOM-killer watches RSS could opt back into eager release.

Rule of thumb for choosing:

• Container with a hard memory cgroup limit, or you're being monitored by RSS-based tooling → MADV_DONTNEED. Kernel accounting reflects reality.
• Hot allocator arena that will be reused within seconds → MADV_FREE. You'll save ~1µs per page in avoided faults.
• Cost math: a minor page fault on Linux is roughly 1000–3000 cycles (~0.5–1.5µs). MADV_FREE avoids that cost entirely on reuse; MADV_DONTNEED guarantees you'll pay it.

Also note: MADV_FREE only works on private anonymous mappings. On file-backed or shared mappings, it returns EINVAL. And on tmpfs (including /dev/shm), MADV_DONTNEED on a shared mapping does not free the underlying page — you need fallocate(FALLOC_FL_PUNCH_HOLE) for that.