2026-08-19
You edit /etc/sudoers at 3 AM. apt-get dist-upgrade rewrites half of /etc the next morning. Somebody's config-management run wipes a hand-tuned sshd_config. Two obvious answers — btrfs snapshots and "just git init in /etc" — both fall over on the same rock: git doesn't record UID, GID, or mode beyond the execute bit, and /etc is full of files where those matter. Check out an old shadow from git and it comes back world-readable. Congratulations, you now have a bigger problem than you started with.
etckeeper is Joey Hess's answer, quietly shipping in Debian since 2007. It wraps git (or hg/bzr/darcs) with metadata tracking, and it hooks into apt/dnf/pacman/zypper so package operations produce automatic before/after snapshots. You get real git log, git blame, git bisect on your system config — with permissions that survive.
sudo apt install etckeeper
sudo etckeeper init -d /etc
sudo etckeeper commit "baseline"
That's it. Now watch:
sudo apt install nginx
# etckeeper hook auto-commits: "committing changes in /etc after apt run"
sudoedit /etc/nginx/nginx.conf
sudo etckeeper commit "bump worker_connections to 4096"
cd /etc && sudo git log --oneline nginx/
# a1b2c3d bump worker_connections to 4096
# 9f8e7d6 committing changes in /etc after apt run
# 3c2b1a0 baseline
The metadata trick lives in /etc/.etckeeper. Peek at it:
sudo head /etc/.etckeeper
# maybe chmod 0640 './sudoers'
# maybe chown 0 './sudoers'
# maybe chgrp 0 './sudoers'
# maybe chmod 0600 './shadow'
# maybe chmod 4755 './fuse.conf'
A pre-commit hook regenerates this file from the live filesystem; a post-checkout-style flow re-applies it. So when you actually roll back:
cd /etc
sudo git checkout HEAD~1 -- sshd_config
sudo etckeeper commit "revert sshd change — broke Ansible"
# mode 0600 and root:root come back correctly
The real payoff is git bisect on a config regression. Node stopped resolving DNS three days ago, twenty commits back?
cd /etc
sudo git bisect start
sudo git bisect bad HEAD
sudo git bisect good HEAD~20
# systemctl restart systemd-resolved && dig example.com
sudo git bisect good # or bad
# ...four steps later git tells you which commit added the bogus nameserver
Some flags worth knowing:
etckeeper unclean — exits 1 if /etc has uncommitted changes. Perfect for a ExecStartPre= in a systemd unit or a nightly check.etckeeper vcs <cmd> — passes through to the underlying VCS with correct working dir and sudo semantics. etckeeper vcs log -p sudoers works.etckeeper pre-install / post-install — the hooks package managers call. Handy to invoke manually when you're doing something equivalent (e.g., a big Chef/Ansible push).etckeeper commit daily by default, catching any manual edit you forgot.Compare with the alternatives. Btrfs/ZFS snapshots capture everything but you can't git log -p a single file, can't git blame, and rollback is filesystem-wide. Plain git init /etc loses permissions the moment you checkout. Config management (Ansible/Chef) is the source of truth for what you intend, but says nothing about what's actually on the disk right now, especially the parts your package manager rewrote behind everyone's back. etckeeper is the audit log for reality.
Extra credit: push /etc to a private remote (git remote add origin git@internal:hosts/$(hostname).git). Now you can diff the actual state of prod against staging from your laptop — without SSHing into either box.
etckeeper turns /etc into a real git repo that survives package upgrades and preserves the file permissions git normally throws away, giving you log, blame, and bisect for the config drift you're currently guessing at.
