2026-08-27
You attach an eBPF program to tcp_sendmsg with fentry and expect it to run every time the function is called. But how? The kernel can't just call your BPF program — it has to save registers, pass arguments, run the program, restore state, and (for fexit) also grab the return value. The answer is the BPF trampoline: a small chunk of JIT-generated x86 that the kernel writes on the fly.
When you attach an fentry program, the kernel:
sub rsp, N, save the six arg registers (rdi, rsi, rdx, rcx, r8, r9) onto the stack, load a pointer to that saved-args area into rdi, then call the BPF program.__fentry__ NOP (a 5-byte NOP placed by -pg -mfentry) into a call trampoline.For fexit, the trampoline additionally calls the original function itself, saves rax (return value) into the args area, then invokes the BPF program. This gives the BPF program access to both arguments and return value — something kretprobes achieve only with a hack (hijacking the return address, adding ~200ns).
Concrete example: bpftrace -e 'fentry:vfs_read { @[comm] = count(); }'. The trampoline for vfs_read(struct file*, char __user*, size_t, loff_t*) is roughly 50 bytes: save 4 args, call BPF, restore, ret. Total overhead per call: ~10ns for fentry, ~15ns for fexit. Compare this to kprobes at ~1000ns (INT3 trap → do_int3 → probe handler → single-step) or the optimized JMP kprobe at ~100ns.
Rule of thumb: If your BTF-typed target function is traceable (has an __fentry__ NOP, isn't marked notrace, isn't in a section ftrace ignores), fentry/fexit is roughly 10× faster than kprobes and gives you typed arguments for free. Use bpftool btf dump file /sys/kernel/btf/vmlinux to check what's available.
The catch: multiple attachments to the same function cascade — the trampoline calls program 1, then program 2, then the original (for fexit). Each attach requires stopping the machine briefly to re-JIT and re-patch the call site atomically via text_poke.
__fentry__ NOP patching plus BTF-derived argument layouts to deliver typed, ~10ns-overhead tracing where kprobes cost 100–1000ns.
