Every modern x86 CPU has a set of Machine Check banks — architectural MSRs (IA32_MCi_STATUS, IA32_MCi_ADDR, IA32_MCi_MISC, IA32_MCi_CTL) exposed per hardware unit: L1D, L2, L3, memory controller, QPI/UPI link, IOMMU, etc. A modern Xeon has 20–32 banks. When a hardware unit detects an error — an ECC-corrected bit flip in DRAM, a parity error on a cache line, a bus timeout, a TLB miscompare — it writes a structured record into its bank's STATUS register.

Two escalation paths exist:

• Corrected Machine Check Interrupt (CMCI): hardware corrected the error (ECC single-bit flip fixed). Raises a threshold-based LAPIC interrupt. The kernel logs it and moves on. This is what feeds edac-util, mcelog, and rasdaemon.
• Machine Check Exception (#MC, vector 18): uncorrectable. The CPU raises an NMI-like exception on the core that touched the bad data. The MCG_STATUS.RIPV bit tells you whether the saved RIP is meaningful (recoverable) or whether the core is basically wedged (PCC = Processor Context Corrupt → panic).

The magic bit is MCi_STATUS.AR (Action Required). If AR=1 and MCi_STATUS.S (Signalled) is set, Linux enters do_machine_check(), decodes the bank, and if it's a memory error with a valid physical address in MCi_ADDR, calls memory_failure() to unmap the offending page and SIGBUS the process that touched it — instead of panicking the box.

Real-world example. Google's 2009 DRAM study across their fleet found roughly 25,000–75,000 correctable errors per Gbit per year, with ~8% of DIMMs experiencing an error in a given year. Meta and Google both published follow-ups showing that a DIMM that logged one CE is dramatically more likely to log an uncorrectable one within 30 days. This is why hyperscalers scrape /sys/devices/system/edac/mc/mc*/ce_count hourly and preemptively drain nodes above threshold. The CE rate is the leading indicator; the UCE is the outage.

Rule of thumb. For an 8-bit SECDED ECC word protecting 64 bits of data, the probability of an undetected error (two bit flips in the same word between scrubs) is roughly C(72,2) × p² where p is per-bit flip probability. At consumer FIT rates (~1000 FIT/Mbit) and a 24-hour scrub interval, that's ~10⁻¹⁵ per word-day — which sounds tiny until you multiply by 10¹² words in a 1 TB server. Enable patrol scrub in BIOS; it walks DRAM in the background and turns latent CEs into logged-and-corrected CEs before two flips collide.

Read your banks with rdmsr 0x401 (MC0_STATUS) through 0x401 + 4×N, or just ras-mc-ctl --summary.