bup: Git's Packfile Format as a Backup Engine, With Rolling Checksums for Sanity

2026-09-12

Everyone's tried to abuse Git as a backup tool. Everyone's watched it explode the first time they committed a 40GB VM image — packing goes quadratic, memory blows up, and there's zero dedup across similar chunks. Meanwhile rsync gives you a mirror, not history; borg/restic work but their chunk stores are proprietary formats you'd have to reverse-engineer if the project ever died.

bup, written by Avery Pennarun around 2010, took Git's packfile format — content-addressable, extensively documented, battle-tested by every developer on earth — and bolted on a rolling-checksum chunker (the same rsync trick) so it can dedupe enormous binary files at the byte level. Every bup repo is a real git repo. In a pinch, git fsck still works on it.

bup init
bup index /var/lib/libvirt/images
bup save -n vmbackup /var/lib/libvirt/images

Run it again tomorrow after 200MB of writes inside a 40GB qcow2. bup rolls a hash across each file, finds chunk boundaries at content-defined points, and stores only the chunks it hasn't seen. A rename or a move? Zero new bytes. A 2MB edit deep inside a huge image? 2MB stored, not 40GB.

Browse and restore feel exactly like Git:

bup ls -l vmbackup/latest
bup restore -C /tmp/out vmbackup/latest/./var/lib/libvirt/images

Or mount the entire history as a filesystem — every snapshot, every version, one FUSE tree:

bup fuse /mnt/bup
ls /mnt/bup/vmbackup/       # every timestamp as a directory

Push to a remote host over ssh, no daemon required:

bup on backup-host save -n laptop $HOME

The wizard tricks are where bup earns its keep:

Watch out for:

Install: apt install bup, brew install bup, or dnf install bup. Python plus a small C extension for the rolling hash.

Key Takeaway: bup marries Git's rock-solid content-addressable storage with a rolling-checksum chunker, so you can snapshot 40GB VM images incrementally, browse every historical version through FUSE, and never worry about the archive format going away.

All newsletters