Correct Backups Require Filesystem Snapshots

databases might need to do lots of writes. (example, could be modify multiple rows in a db). They can't neccessarily capture it into a single write operation. they will issue multiple.

VSS is a method windows has to allow application to both be informed that a snapshot operation is in progress (and should quiesce their writes) as well as the ability to delay the snapshot operation from occurring (because they are in the middle of a "transaction"). If they delay too much, the snapshot operation will fail and reported back as such to the caller.

edit: also see the former maintainer of VSS at MSFT who responded to me in my other post in this thread

This article also fails to consider that on modern systems writes are often batched and not immediate. The only way to get an actually consistent and safe backup of any file / file system, is to actually shut down the machine so that all applications, and the OS, have shut down gracefully.

As an intermediate step, it may be practical to consider shutting down your application, flush all writes (e.g. via sync command), and taking the file system snapshot then, restarting the application, and then taking the backup from that file system snapshot. At least then your critical data should have a consistent and safe backup, even if the rest of the OS _may_ be in a suspect state.

I wrote https://github.com/Freaky/zfsnapr a few months ago so I could finally have point-in-time consistent Borg backups with ZFS snapshots, without having the mess of teaching Borg where every .zfs directory was.

It recursively snapshots mounted pools, and recursively mounts snapshots of the mounted datasets into a target ready to point your backup tools at. I do so via a chroot so I didn't need to make any changes to my Borg setup - just to how I run it.

A system snapshot for a personal computer is potentially a really difficult thing to restore (ex. you break the device and need a new one, unless it matches exactly restoring snapshot likely puts you in OS recovery mode).

A volume snapshot if you're technical enough to split up OS and application data volumes on your local drive can potentially help but you still have registry or other issues.

The best way to handle backups, from the perspective of a backup company CEO, is to backup your critical data, and that could include application configurations.

Browsers support exporting your profile, good backup software let's you run a pre-backup job script and post-job script, so you can export your browser profile, lock certain files or folders during a backup, etc. to get a good recovery point while minimizing risk of data corruption.

Way longer topic than a comment here. If you do want full system state, pick a hypervisor (Parallels, etc.) and then backup the full VM each time you power the VM down. It'll make for much larger backups so restores are slower and storing all the recovery points will cost you a bit more too.

You write "transaction", but Unix filesystems don't know anything about any transactions from the application's point of view. To them it's just a bunch of write(2) calls. How is a ZFS snapshot going to make sure that it snapshots the state either before or after what an application considers a "transaction", but implements with a sequence of write(2), fsync(2) etc. calls? (EDIT: The rest of this paragraph is rescinded. See [1]) If an application uses SQLite, BerkeleyDB etc., then you don't need ZFS. XFS will be sufficient (and maybe even better?). If an application doesn't use any transaction-processing library ("database"), then I don't see how filesystem snapshots fix anything here --- you still can get an intermediate state in a snapshot.

As far as I understand, the only thing snapshots make sure of is that you're looking at a consistent state of a filesystem from the point of view of the filesystem, so e.g. if there was a sequence of operations "remove file a/b", "create file c/d", then you're not going to record a state with both files "a/b" and "c/d" present --- something that could happen with a backup tool not using filesystem snapshots, if it reads directory "a" first and directory "c" second. But there are no assurances about application transactions. So if KeePassXC has a bug, then if there is a sequence of operations "remove password A", "add password B", then a backup tool may pick up a version of KeePassXC database which is going to contain both passwords A and B, even if the backup tool uses filesystem snapshots.

[1]: At first I didn't get the example from TFA about Firefox and SQLite, but I've reconsidered and it makes sense now. I'll still say that you may get an intermediate state in a snapshot. However, snapshot prevents some classes of inconsistencies (though not all). Call me convinced about ZFS.