2026-09-03
When code runs in NMI (Non-Maskable Interrupt) context, it faces a brutal constraint: it can preempt anything, including code holding spinlocks the NMI handler might want. Take a lock in an NMI? Instant deadlock if the interrupted code already held it. Call printk()? It grabs the console lock. Wake a task? Scheduler locks. The NMI handler is effectively forbidden from touching most of the kernel.
The irq_work subsystem (kernel/irq_work.c) solves this. It lets NMI (or any hardirq) context enqueue a callback that runs shortly afterward in a safer context — typically at the tail of the next interrupt, after locks would have been released.
How it works: Each CPU has two lock-free lists: raised_list (needs immediate self-IPI) and lazy_list (runs on the next tick). You allocate a struct irq_work, set a callback function, and call irq_work_queue(). The subsystem uses cmpxchg to atomically push onto the per-CPU list, then triggers a self-IPI via the APIC. When that IPI fires, irq_work_run() drains the list and invokes each callback — now in a normal hardirq context where sleeping is still forbidden, but console locks, scheduler wakeups, and most spinlocks are safe.
Real-world example: perf sampling. When the PMU overflow interrupt fires as an NMI (because you're profiling kernel code that runs with interrupts disabled), the handler needs to wake up the userspace perf process reading the ring buffer. But wake_up() grabs the runqueue lock — a spinlock. From NMI context, that's a potential deadlock. The solution: the NMI handler writes the sample into the ring buffer (lock-free by design), then enqueues an irq_work whose callback calls wake_up() from safe context. Same pattern for printk() from NMI — the message goes into a lock-free log buffer, and an irq_work flushes it to the console later.
The cost: Enqueuing is roughly one cmpxchg plus an IPI (~1000 cycles for the self-IPI round trip on modern x86). The callback runs within microseconds. Compare that to attempting the work in NMI directly, which risks locking up the entire machine.
Rule of thumb: If your handler runs in a context where in_nmi() returns true, you can only touch: (1) percpu variables, (2) lock-free structures, (3) atomic operations, and (4) irq_work_queue(). Everything else — printk, wakeups, allocations, most locks — must be deferred. The irq_work indirection typically adds under 10 microseconds of latency, which is negligible compared to debugging a kernel hang caused by an NMI-context deadlock.
