2026-05-26
The original APIC was designed when "many cores" meant 4. Its destination field is 8 bits, which caps the system at 256 logical CPUs — and in practice, far fewer once you reserve broadcast and cluster encodings. By the time AMD shipped 128-core EPYCs and Intel shipped 60-core Xeons with hyperthreading, the legacy xAPIC was an interrupt-routing time bomb.
x2APIC is the fix. Three things change:
wrmsr to register 0x800–0x8FF, which is roughly 3x faster than the MMIO path because there's no cache line ping-pong on the APIC page.Why cluster mode matters: In flat physical mode, sending an IPI to "all cores except me" on a 128-core box means 127 individual ICR writes. Each write stalls until the previous IPI is accepted. In x2APIC cluster mode, you send one IPI per cluster (typically 8 clusters of 16 cores), so the same broadcast becomes 8 writes — a 16x reduction in IPI emission latency. This is why Linux's flush_tlb_others() got dramatically faster around kernel 3.x when x2APIC cluster mode landed.
Real example: A TLB shootdown on a 64-core machine. In legacy xAPIC, the initiating CPU writes ICR 63 times, waiting ~100ns per send for delivery acknowledgment — about 6.3µs just emitting IPIs, before any target even handles the flush. In x2APIC cluster mode with 4 clusters of 16, it's 4 writes, ~400ns. The flush itself still takes microseconds, but the emission phase nearly disappears from the profile.
Rule of thumb: If you see cpu_flags: ... x2apic ... in /proc/cpuinfo and dmesg | grep x2apic shows "x2apic enabled," your IPI broadcasts are O(clusters), not O(CPUs). On a 96+ core machine without x2APIC (rare, but possible via BIOS misconfiguration), TLB-heavy workloads can lose 5–10% throughput purely to IPI serialization.
Gotcha: x2APIC requires interrupt remapping (the IOMMU's IR unit) to be enabled, because the 32-bit ID doesn't fit in legacy MSI message formats. Disabling VT-d in BIOS silently drops you back to xAPIC and caps you at 255 CPUs — which is why some servers refuse to boot all cores until IOMMU is on.
