The Workqueue and kworker Threads: Why Some Kernel Work Runs in a Schedulable Context

2026-06-30

Softirqs and tasklets run in atomic context — they can't sleep, can't block on a mutex, can't call any function that might schedule. That's fine for trivial packet handoff, but a lot of kernel work needs to wait: allocating with GFP_KERNEL, taking a sleeping lock, doing I/O. The workqueue subsystem is the kernel's answer. You queue a work_struct, and a kworker kernel thread picks it up and runs it in process context, where sleeping is legal.

Modern Linux uses concurrency-managed workqueues (CMWQ). Instead of one thread per workqueue (the old model that spawned hundreds of mostly-idle threads), CMWQ maintains shared worker pools per CPU. Each CPU has two pools: a normal-priority pool and a high-priority pool. Workers in a pool service any workqueue bound to that CPU. The pool watches its workers — if every worker is sleeping (blocked on I/O, a mutex, etc.) and there's still pending work, it spawns or wakes another worker. That's the "concurrency-managed" part: thread count tracks runnable work, not queued work.

You'll see them in ps as [kworker/0:1-events] — CPU 0, worker ID 1, currently servicing the events workqueue. The -H suffix means high-priority pool; u (e.g. kworker/u16:2) means unbound — not pinned to a CPU, scheduled wherever the load balancer puts it. Unbound pools matter for work that's CPU-intensive or memory-bandwidth-heavy, where the scheduler should be free to migrate.

Real example: A USB device disconnect fires an interrupt. The IRQ handler can't free the device's data structures — that involves sleeping locks and waiting on in-flight URBs to drain. So it queues work to the usb_hub_wq workqueue. A kworker picks it up, sleeps as needed while teardown runs, and exits. Same pattern for filesystem writeback, async block I/O completion, and most driver deferred work since ~2010.

Rule of thumb: If your deferred work might sleep, use a workqueue. If it absolutely cannot sleep and must run before the next interrupt, use a tasklet or softirq. The cost: workqueue dispatch adds ~1–5 µs of scheduler overhead versus <100 ns for a tasklet. For a 1 Gbps NIC processing a packet every 12 µs, that's a 30%+ tax — which is why network RX uses softirqs, not workqueues.

Check /sys/devices/virtual/workqueue/ to see active queues, their CPU affinity, and per-queue tuning knobs like max_active (caps simultaneous executions of work from one queue).

Key Takeaway: Workqueues let kernel code defer work to a schedulable process context where sleeping is legal, with CMWQ dynamically sizing per-CPU worker pools to match actual concurrency rather than queued volume.

All newsletters