2026-06-17
You queued a few ffmpeg encodes with nohup &, logged out, came back, and now you have no idea which finished, which crashed, or which is still chewing CPU. bg/disown hides them. tmux panes scale to about four before you give up. make -j requires a Makefile. GNU parallel is one-shot — fire and pray.
pueue (Rust, by Arne Beer) is the missing piece: a persistent task-queue daemon with parallel slots, groups, pause/resume, per-task working directories and envs, JSON status, and live output streaming. Think of it as a CI runner for your own laptop.
Get it going:
cargo install pueue # or: brew install pueue, apt install pueue
pueued -d # daemon; better: enable as a systemd --user unit
Now queue work and forget about it:
pueue add -- yt-dlp 'https://...'
pueue add -- ffmpeg -i in.mkv -c:v libx265 -crf 22 out.mkv
pueue add -- rsync -aHAX /data backup:/data
pueue status # table of id, status, command, runtime
pueue log 3 --lines 50 # last 50 lines of task 3's combined output
pueue follow 3 # tail -f the running task
The parts that make it stop being a toy:
pueue parallel 4 sets how many tasks run at once. Bump it down when you start gaming, back up overnight.pueue group add net --parallel 8
pueue group add cpu --parallel 2
pueue add -g net -- aria2c "$URL"
pueue add -g cpu -- ffmpeg -i in.mkv -c:v libx265 out.mkv
pueue pause 7 sends SIGSTOP; pueue start 7 sends SIGCONT. Free RAM mid-encode, then resume.pueue add --after 12 13 -- ./publish.sh won't run until both upstream tasks succeed.--working-directory, --label foo, --escape, and an interactive --env editor. No shell-quoting roulette.--json:
# Restart only the failed tasks from the last hour
pueue status --json \
| jq -r '.tasks | to_entries[]
| select(.value.status == "Failed") | .key' \
| xargs pueue restart -i
pueue wait blocks until everything finishes — useful in shell scripts that fire dozens of tasks and need a join point.Compared to nq (covered earlier), the trade is intentional: nq is just files in a directory and dies if you do. pueue wants a daemon, but in return you get state that survives reboots (the queue is serialized to disk), priorities, groups, signals, structured logs, and a query interface. For a half-dozen background fetches, nq wins on simplicity. For 200 encodes overnight across two resource pools where you want to retry the failures in the morning, pueue is the only sane answer that doesn't involve writing your own systemd templates.
One last trick most people miss: pueue send <id> "y\n" pipes input into a running task's stdin. That alone saves you from rewriting an interactive script just to batch it.
& and nohup but don't warrant a real scheduler, pueue gives you a persistent, group-aware, scriptable task daemon that treats your shell tasks like first-class jobs.
