Compilers emit instructions one at a time, but modern CPUs are smarter than that. Instruction fusion is the hardware trick where the front-end detects pairs of adjacent instructions that always travel together and merges them into a single micro-op (µop) that flows through the pipeline as one entity. Fewer µops means less pressure on the reorder buffer, fewer scheduler entries consumed, and one retirement slot instead of two.

There are two flavors, and they happen at different stages:

• Macro-op fusion: combines two architectural instructions before decode. The canonical example is cmp + jcc (compare followed by conditional jump). Intel has done this since Core 2; AMD since Bulldozer. The decoder sees cmp rax, rbx followed by jne label and emits one fused compare-and-branch µop.
• Micro-op fusion: combines two internal µops from a single complex instruction (like a load-op pair from add rax, [rbx]) so they share a single ROB entry but still dispatch to two execution ports.

Concrete example. Consider a tight loop counter:

loop:
  dec rcx
  jnz loop

Without fusion, that's 2 µops per iteration. With macro-op fusion (which works for dec/jcc, sub/jcc, and/jcc, test/jcc, cmp/jcc), it becomes 1 fused µop. On a 4-wide retire CPU, this doubles your effective loop throughput from 2 iterations/cycle to 4 iterations/cycle — assuming the branch predictor cooperates.

Rule of thumb. If your hot loop's IPC seems suspiciously high (say, 5 IPC on a 4-wide machine), fusion is the explanation. Measure UOPS_RETIRED.MACRO_FUSED via perf: perf stat -e uops_retired.macro_fused ./your_program. Ratio of fused to total retired µops above ~10% is common in branch-heavy code.

Constraints that bite.

• The two instructions must be adjacent in the instruction stream — a stray nop between them kills fusion.
• Not all jcc conditions fuse on all CPUs. Older Intel parts only fused unsigned conditions with cmp/test.
• The pair must fit within a single 16-byte (or 32-byte) fetch window on some microarchitectures.
• Memory operands in the arithmetic instruction disable macro-op fusion on most x86 designs.

This is why hand-rolled assembly that schedules cmp far from its jcc "for ILP" is usually slower than the naive adjacent form — you've forfeited a free µop.