Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Time Machine-style backups with rsync (2018)

samuelhewitt.com

Post reply on HN
Print view View original

21–30 of 57 posts

Re: Time Machine-style backups with rsync (2018)

#21

Post by haradion »

locknitpicker wrote: post #18
LeoPanthera wrote: post #17

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are,…

It doesn't really hurt in practice because it's only one part of the full backup procedure. Deduplicate to save space; re-duplicate the (smaller) backups to separate media for redundancy; scrub regularly for bit rot and rotate your media. A proper backup system requires all but the first of those anyway.

Re: Time Machine-style backups with rsync (2018)

#23

Post by stuxnet79 »

locknitpicker wrote: post #18
LeoPanthera wrote: post #17

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are,…

If you are storing your data in a filesystem like ZFS then the risk is lower since you'll know you have a bad sector long before you attempt to deduplicate. But I otherwise share your same concerns and I'm eager to hear from others how to mitigate this. I've lost a volume due to poor practice while encrypting so I'm understandably reluctant to overcomplicate my backup strategies in the name of space compression or privacy.

Re: Time Machine-style backups with rsync (2018)

#25

Post by movetheworld »

mrtesthah wrote: post #2

Isn’t restic better for backups overall? Anyone have a good script for macOS triggered by launchd, ideally something that uses FSEvents to check for directory changes?

I use restic, too, and I am very pleased with it, although I run it on windows, which works, but isn't perfect. So, you use it regularly?

Re: Time Machine-style backups with rsync (2018)

#27

Post by DaSHacka »

NSUserDefaults wrote: post #19
locknitpicker wrote: post #18

Earlier quoted context omitted.

> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are,…

In that case you are supposed to use your /other/ backup. Which you have.

Defeating the point of deduplication...

Re: Time Machine-style backups with rsync (2018)

#28

Post by 00deadbeef »

EGreg wrote: post #8

Why make hard links when you can use rsync or syncthing to just make an actual copy on an external hard drive eg via wifi or just remotely?

It does make an actual copy but then it builds a directory structure that you can browse by date (like Time Machine). That directory contains hard links so only one copy of a file is ever backed up but you see contents that match the date of the backup.

Re: Time Machine-style backups with rsync (2018)

#29

Post by LeoPanthera »

locknitpicker wrote: post #18
LeoPanthera wrote: post #17

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

> "borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. Deduplication helps minimize space, but isn't it a major liability in backups? I mean, what happens when you try to restore your backups but a lone sector holding a file from way back in the past happens to not be recoverable? Doesn't it mean that no matter how frequent your backups are,…

This is bordering on paranoia. If the bad sector contains a critical part of the filesystem, you're going to lose everything anyway.

Do modern disks even have physical "sectors" anymore? Isn't it all virtual?

Without dedup you're just going to backup less stuff, which is far worse.

Re: Time Machine-style backups with rsync (2018)

#30

Post by IshKebab »

LeoPanthera wrote: post #17

"borg" has basically solved backups permanently. It's deduplicated, even across snapshots, compressed, and end-to-end encrypted. I'm surprised it's not more well known.

I used to use Borg, but Restic (and it's Rust clone Rustic) are better.
Post reply on HN