2026-07-05
A 100Gbps NIC delivers ~8 million packets per second. No single core can process that. Receive Side Scaling (RSS) is the hardware mechanism that distributes incoming packets across multiple receive queues, each pinned to a different CPU, so the load fans out before software ever sees it.
The mechanism has three parts:
/proc/irq/N/smp_affinity to a specific CPU. That CPU runs the softirq, allocates the skb, and pushes it up the stack.Why the indirection table matters: Without it, you'd have hash-mod-N queue assignment, and changing N (say, disabling a core) would reshuffle every flow. With RETA, you rewrite one 128-byte table and existing flows stay on their queues — critical for TCP, where reordering triggers spurious retransmits and cuts throughput in half.
Concrete example: On a 32-core server running an nginx reverse proxy, ethtool -x eth0 shows the current RETA. A common tuning is to restrict RSS to cores on the same NUMA node as the NIC's PCIe slot. If your NIC is on socket 0 (cores 0–15), you rewrite RETA so all 128 entries point only at queues 0–15:
ethtool -X eth0 equal 16
This avoids cross-socket QPI/UPI traffic for every packet — a single cross-socket cache line miss is ~200 cycles vs. ~40 for local L3.
Rule of thumb: Number of RX queues = number of cores on the NIC's NUMA node, not total cores. Adding queues past that point adds interrupt overhead without adding throughput, because you've saturated the memory bandwidth path from NIC to those cores.
Gotcha: RSS hashes the 4-tuple, so all packets in one TCP flow land on one queue — one core. A single elephant flow can't be parallelized by RSS alone. That's what Receive Packet Steering (RPS), the software cousin, exists to fix: it re-hashes in the kernel and redistributes to other cores via IPI.
