When Linux exhausts memory and reclaim fails, it doesn't return ENOMEM to your allocator — it picks a victim process and sends SIGKILL. The mechanism is the OOM killer, and the victim is chosen by a score computed in oom_badness() at mm/oom_kill.c.

The base score is the process's resident memory footprint, in pages:

• RSS (anonymous + file-backed mapped)
• Plus swap usage
• Plus pagetable pages

This sum is normalized against total available memory and scaled to 0–1000. A process using 30% of RAM gets ~300. Higher score = more likely to die. The kernel iterates all eligible tasks, picks the highest score, and kills it (along with all threads sharing its mm_struct).

Userspace can bias the decision via two files in /proc/<pid>/:

• oom_score_adj: integer in [-1000, +1000], added directly to the score. -1000 makes a process unkillable; +1000 makes it the preferred victim.
• oom_score: read-only, the current computed value.

Real-world example: systemd sets OOMScoreAdjust=-900 on sshd so you can still log in after an OOM event. Conversely, Kubernetes sets oom_score_adj=+1000 on BestEffort pods, +999 down to ~+2 on Burstable pods (scaled by requested memory), and a negative value on Guaranteed pods. That's literally how QoS classes get enforced under memory pressure — not by cgroup limits alone, but by biasing who the OOM killer eats first.

Rule of thumb: if your process is using X% of RAM and has oom_score_adj = A, its score is roughly 10·X + A. A process at 20% RAM with adj=+200 (score ~400) will be killed before one at 35% RAM with adj=0 (score ~350).

Two gotchas:

• The killer runs in the context of the allocating task, holding oom_lock. If the chosen victim doesn't release memory fast enough (e.g., stuck in uninterruptible I/O), the system livelocks — this is why vm.oom_kill_allocating_task=1 exists as an escape hatch: kill the requester, no scan.
• Memory cgroups have their own OOM killer scoped to the cgroup. A container hitting its memory.max triggers cgroup OOM, which uses the same scoring but only considers tasks in that cgroup — the host doesn't notice.

Check dmesg after a mysterious process death: the kernel logs the full task list with scores, RSS, and the victim's PID. It's the fastest forensic trail you have.