Database backups, dump files and restic

Backup Wars: ditching dump files sparks Restic vs Borg brawl

TLDR: Author ditched daily database dump files for streaming backups with restic, cutting clutter and risk. Comments split: one fan touts borg for giant, mostly unchanged data, while others praise restic’s simplicity—turning a dry backup tweak into a mini showdown over which tool fits real-world data.

Backups got spicy! The author admits their old routine dumped databases into a daily compressed file—then watched it change even when nothing in the data did. Cue backup bloat and a side of drama: metadata changing, “rsyncable” flags, and an insecure temp-file oops. Enter the glow-up: streaming straight into restic so nothing lands on disk, with built‑in deduplication (removes duplicates), compression (shrinks it), and encryption (locks it). Bonus nerdy twist: a quirky bug means the “stdin filename” can’t include a slash, but hey, it works link.

The crowd quickly split into factions. Team Restic cheered the clean, pipe-it-right approach, calling dump files “Groundhog Day backups.” Team Borg rolled in with receipts: gmuslera swears borg thrives on giant, mostly-unchanged databases, where new dumps look like old dumps and dedup sniffs out repeated blocks like a truffle pig. The strongest opinion? Use the tool that matches your data’s reality—restic for simplicity and mobility, borg for heavy, historical archives.

Humor flew fast: “rsyncable therapy,” “dump files got dumped,” and “metadata mood swings.” Amid the meme storm, timwis dropped a wholesome “learned a few things,” while others debated whether flags and filesystems are magic or myth. The vibe: practical tips with a pinch of backup soap opera.

Key Points

  • The old backup method wrote daily mysqldump outputs to a gzip-compressed file, which duplicated data on disk.
  • File-based dumps changed even without data modifications due to metadata differences, causing rsnapshot to store extra copies.
  • Mitigations included using btrfs with reflinks and gzip’s --rsyncable flag, which helped but didn’t fully solve inefficiencies.
  • The new method streams mysqldump directly into restic via --stdin-from-command, avoiding intermediary files and leveraging deduplication, compression, and encryption.
  • The script uses consistent options (e.g., --skip-dump-date, --single-transaction) and avoids a restic bug by keeping --stdin-filename free of “/”.

Hottest takes

"I moved the big DB to borg—new dumps are mostly identical, dedup eats it for breakfast" — gmuslera
"Thanks for sharing—definitely learnt a few things" — timwis
Made with <3 by @siedrix and @shesho from CDMX. Powered by Forge&Hive.