Every non-inlined function in a modern Linux kernel starts with a curious sequence: a 5-byte NOP. That NOP is not laziness or padding — it is a reservation. It exists so ftrace can patch in a call __fentry__ at runtime and turn any function into a traced function without recompiling, rebooting, or measurable overhead when disabled.

Compile the kernel with -pg -mfentry (GCC) and the compiler emits, as the very first instruction of every function, a call to __fentry__ — before the prologue, before push %rbp, before anything. On x86-64, that call is a 5-byte E8 relative-call encoding. At boot, ftrace walks the __mcount_loc section (a table the linker built listing every one of those call sites — often 50,000+ entries on a distro kernel) and rewrites each one to 0F 1F 44 00 00, the recommended 5-byte NOP. Cost when tracing is off: one decoded NOP the CPU folds into nothing. Measured overhead: below the noise floor.

Enable a tracer via /sys/kernel/tracing/current_tracer and ftrace reverses the patch — atomically, using the INT3 breakpoint trick. It writes 0xCC over the first byte, IPIs every CPU to serialize instruction fetch, writes bytes 2-5, then flips byte 1 back to E8. Any CPU that traps on the INT3 mid-patch gets bounced through a handler that emulates the call. No stop_machine(), no downtime.

Concrete example: to see every function called during a single ls:

• echo function_graph > /sys/kernel/tracing/current_tracer
• echo 1 > /sys/kernel/tracing/tracing_on; ls; echo 0 > ...
• cat /sys/kernel/tracing/trace — you get an indented call graph with per-function nanosecond durations, courtesy of __fentry__ on entry and a return trampoline that hijacks the return address on the stack.

Rule of thumb: the nop sled costs ~5 bytes × function-count. A kernel with 50,000 functions burns ~250KB of .text on tracing hooks it may never use. In exchange, you get whole-kernel dynamic tracing with zero recompile. The ratio — 250KB of icache pollution for infinite observability — is why every serious kernel ships with it enabled.

The return-side trick is worth noting: the graph tracer replaces the caller's return address on the stack with a pointer to return_to_handler, saves the real address in a per-task shadow stack, times the function, then jumps to the real return. This is exactly what CET's shadow stack was designed to catch — so ftrace and CET need explicit reconciliation to coexist.