11–20 of 27 posts
What happens if in the process of all this read write rewrite, data is corrupted?
It works. Already implemented: https://rdiff-backup.net/ https://github.com/rdiff-backup/rdiff-backup There are also other tools which have implemented reverse incremental backup or backup with reverse deduplication which store the most recent backup in contiguous form and fragment the older backups.
It sounds like this method is I/O intensive as you are writing the complete image at every backup time. Theoretically, it could be problematic when dealing with large backups in terms of speed, hardware longevity, and write errors, and I am not sure how you would recover from such errors without also storing the first image. (Or I might be misunderstanding your idea. It is not my area.)
The low likelihood / high impact edge case this does not handle is: "Oops, our data center blew up." An extreme scenario, but one that this method does not handle. It instead turns your most recent backup into a single point of failure because you cannot restore from other backups.
For reference: a comprehensive backup + security plan for individuals https://nau.github.io/triplesec/
Earlier quoted context omitted.
No, they work the opposite way using copy-on-write.
"For files that changed or were deleted: move them into a new delta folder. For new/changed files: copy them into the latest snapshot folder." is just redneck copy-on-write. It's the same result but less efficient under the hood.
I used to work on backup software. Our first version did exactly that. It was a selling point. We later switched approach to a deduplication based one.
Yes, the deduplicated approach is superior, if you can accept requiring dedicated software to read the data or can rely on a file system that supports it (like Unix with hard links).
I'm looking for a cross-platform solution that is simple and can restore files without any app (in case I didn't maintain my app for the next twenty years).
I'm curious if the software you were working on used proprietary format, was relying on Linux, or used some other method of duplication.
The more common approach now is incrementals forever with occasional synthetic full backups computed at the storage end. This minimises backup time and data movement.
According to chat GPT the forward delta approach is common because it can be implemented purely append only, whereas reverse deltas require the last snapshot to be mutable. This doesn't work well for backup tapes.
Do you also think that the forward delta approach is a mere historical artifact?
Although perhaps backup tapes are still widely used, I have no idea, I am not in this field. If so the reverse delta approach would not work in industrial settings.
The more common approach now is incrementals forever with occasional synthetic full backups computed at the storage end. This minimises backup time and data movement.
I agree it seems more common. However back-up time and data movement should be equivalent if you follow the algo steps. According to chat GPT the forward delta approach is common because it can be implemented purely append only, whereas reverse deltas require the last snapshot to be mutable. This doesn't work well for backup tapes. Do you also think that the forward delta approach is a mere historical artifact? Altho…
This is more-or-less how most cloud backups work. You copy your “premium” SSD to something like a shingled spinning rust (SMR) that behaves almost like tape for writes but like a disk for reads. Then monthly this is compacted and/or archived to tape.
[1] For some values of nobody.