You've hit this wall: tail -f /var/log/app.log | grep ERROR | awk '{print $5}' sits there silently for ages, then dumps a wall of output at once. Your filter is working — it's just that glibc switched grep to 4KB block buffering the moment it noticed stdout was a pipe instead of a terminal. So lines pile up in libc's internal buffer until the buffer fills, the program exits, or the heat death of the universe.

Most people "solve" this by Googling, finding "use grep --line-buffered," and walking away. That works for grep, sed (-u), and awk (fflush()). It does not work for the random binary you didn't write and can't recompile. Enter stdbuf, shipped with GNU coreutils, which has been hiding under your nose since 2009.

# Force line-buffering on stdout and stderr for ANY glibc program
stdbuf -oL -eL my-noisy-tool | grep WARN

# The classic "make the pipeline actually stream"
tail -f access.log | stdbuf -oL grep 500 | stdbuf -oL cut -d' ' -f7 | uniq

# Fully unbuffered (slower, but every byte is flushed)
stdbuf -o0 some-binary | tee output.log

# 1MB buffer instead of the default — useful for high-throughput pipes
stdbuf -o1M heavy-emitter | downstream-aggregator

The flags map cleanly: -i stdin, -o stdout, -e stderr. Each takes 0 (none), L (line), or a size like 4K, 1M.

How the magic actually works: stdbuf sets LD_PRELOAD=libstdbuf.so plus a few env vars. When the target binary starts, libstdbuf's constructor runs and calls setvbuf(3) on the requested streams before main() ever executes. There is no kernel involvement, no ptrace, no patching. It's the cleanest LD_PRELOAD hack in the standard toolbox.

That mechanism is also its limitation. stdbuf only works if:

• The program uses glibc stdio (FILE* / fprintf / puts) for the affected stream. Programs that write(2) directly to fd 1 ignore it.
• The program doesn't override buffering itself. Python re-runs setvbuf internally — use python -u or PYTHONUNBUFFERED=1. Same story for Go's bufio.Writer.
• It's dynamically linked. Static musl binaries laugh at your LD_PRELOAD.
• It's not setuid. The dynamic linker strips LD_PRELOAD for security.

For the cases where stdbuf doesn't work, the escalation ladder is:

• unbuffer from the expect package — allocates a PTY, so the target program thinks it's talking to a terminal and uses line buffering naturally. Works on programs that explicitly check isatty() and refuse to play otherwise (looking at you, ls --color, grep --color).
• script -qfc 'your-cmd' /dev/null — same PTY trick using util-linux.
• socat - EXEC:'your-cmd',pty,raw,echo=0 — when you need bidirectional PTY plumbing.

The diagnostic move: if a pipeline mysteriously stalls or batches, run the producer alone in a terminal. If output streams fine to the TTY but not through a pipe, you've got a buffering problem, and stdbuf -oL is your three-second fix before you go reaching for ptrace.

Bonus party trick — combine with ts from moreutils to timestamp actually streaming output: stdbuf -oL ./long-job | ts '%H:%M:%.S'. Without stdbuf, every timestamp would be wrong by however long it took to fill 4KB.