Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Best Centralized Backup Solution

news.ycombinator.com

Post reply on HN
Print view View original

41–50 of 70 posts

Re: Ask HN: Best Centralized Backup Solution

#41

Post by saltcured »

yjftsjthsd-h wrote: post #14
zmix wrote: post #9

burp ( https://burp.grke.org/ ) comes to mind, as well. Windows and UNIX-like only, no macOS.

"For Mac users, burp is available in Homebrew. See the quick start page for more information." - https://burp.grke.org/download.html Also, I can't imagine how something could be for Unix-likes and not target Mac OS, since it is literally a certified Unix.

Writing a backup tool and supporting an OS properly is a lot more work than just writing portable software that can run on the portable standard APIs supported by an OS. People expect their backups to preserve file attributes, access control policies, etc.

Beyond traditional POSIX file attributes like ownership and access mode, there are POSIX ACLs. Linux also has extended attributes and SELinux policies. Windows has a different ACL model, and NFSv4 ACLs on Linux are more like those. A Mac has "resource forks" which have no real analogue in Linux or POSIX systems, and a backup/restore that ignores these would be very useless to most Mac users.

Re: Ask HN: Best Centralized Backup Solution

#42

Post by rmu09 »

m_b wrote: post #5

BackupPC is the best one: https://backuppc.github.io/backuppc/ You get every professional features out of the box (full/inc backups, deduplication, compression...) & everything is automated.

backuppc works really well, the drawback is that it is kind of slow if you have lots (millions) of files.

Re: Ask HN: Best Centralized Backup Solution

#43

Post by rmu09 »

espadrine wrote: post #36
bagsvaerd70 wrote: post #4

I like to rsync into a BTRFS volume, and then take a snapshot. It's simple, fine-grained (you can use .rsyncignore files like .gitignore files) and versioned.

Why do you choose not to use Btrfs' send and receive commands? They do what rsync does, a bit more efficiently, unless I am missing something. http://marc.merlins.org/perso/btrfs/post_2014-03-22_Btrfs-Ti...

btrfs send will only work on btrfs filesystems, you probably also want to backup other stuff.

Re: Ask HN: Best Centralized Backup Solution

#44

Post by rolleTx »

type0 wrote: post #35
catwell wrote: post #15

Earlier quoted context omitted.

No idea why this was downvoted. I don't have recent experience setting up something like this but 10 years ago BackupPC was the answer, and I guess it is still good. Does anyone have a different experience?

> No idea why this was downvoted Perl?

How can you downvote a reply?

Re: Ask HN: Best Centralized Backup Solution

#45

Post by wetpaste »

m_b wrote: post #5

BackupPC is the best one: https://backuppc.github.io/backuppc/ You get every professional features out of the box (full/inc backups, deduplication, compression...) & everything is automated.

I have used backupPC for quite a while. It works great but doesn't scale very well when you are getting to the scale of TBs upon TBs of files, 100s of millions of files. I ended up having to run multiple servers.

If you want something along these lines that scales better, rsnapshot works wonders. Rsnapshot is much more simple, as it doesn't create a large pool of files to compare and de-dupe against, it uses rsync's built in deduplication features (so, it doesn't create a "pool" like backuppc.) Techincally, backuppc uses less storage space, so if that's the concern, use it. but rsnapshot is my tool of choice.

EDIT: I will mention that my experience is based on having to deal with very large sets of data across numerous backup targets and at least 4 years ago, I have not tried any of the latest updates that may make this solution better or make performance better. Most people that were in my position probably would have had the funds to leverage an enterprise vendor solution at that scale. For what backupPC does, it was pretty amazing to work with and rock solid in terms of functionality at smaller scales.

Re: Ask HN: Best Centralized Backup Solution

#47

Post by __d »

espadrine wrote: post #36
bagsvaerd70 wrote: post #4

I like to rsync into a BTRFS volume, and then take a snapshot. It's simple, fine-grained (you can use .rsyncignore files like .gitignore files) and versioned.

Why do you choose not to use Btrfs' send and receive commands? They do what rsync does, a bit more efficiently, unless I am missing something. http://marc.merlins.org/perso/btrfs/post_2014-03-22_Btrfs-Ti...

For my usage, I have BTRFS on my backup host, but not on all (or even any) of the machines I want to back up. They're variously macOS, Windows, Linux ext4, FreeBSD, and OpenBSD.

rsync has the upside of being universal.

Re: Ask HN: Best Centralized Backup Solution

#48

Post by rasengan »

pmarreck wrote: post #38
rasengan wrote: post #16

Earlier quoted context omitted.

If it is targeting UNIX then it must work on Mac. Echoing your words, UNIX has stringent requirements to be UNIX and macOS fulfilled those requirements. MacOS IS UNIX. As a side note for fun, Linux is not UNIX and that’s intentional. Freedom is nice. ;)

> Linux is not UNIX and that’s intentional Isn't that the recursive acronym itself? :)

Exactly!

Re: Ask HN: Best Centralized Backup Solution

#49

Post by mercora »

espadrine wrote: post #36
bagsvaerd70 wrote: post #4

I like to rsync into a BTRFS volume, and then take a snapshot. It's simple, fine-grained (you can use .rsyncignore files like .gitignore files) and versioned.

Why do you choose not to use Btrfs' send and receive commands? They do what rsync does, a bit more efficiently, unless I am missing something. http://marc.merlins.org/perso/btrfs/post_2014-03-22_Btrfs-Ti...

In addition to what has already been said, the last time i used btrfs in this manner the protocol had issues with some extended attributes which happened to manifest in issues on binaries which lets them gain additional capabilities[0] for example. Most notably this made iputils ping[1] utility unusable after branching of a previously made snapshot for further usage. This appears not to be a too common thing in other distributions but in arch the ping utility uses CAP_NET_RAWIO capability in order to avoid the need to use setuid which appears to be more common elsewhere. None the less i think that is something one would want to know the consequences of as i am sure there are quite a few more binaries with extended capabilities set in the extended attributes of its file or use them in another manner like security labels for SELinux. I just rechecked if this is still the case and found this patch[2] but i am not sure this has already landed.

[0] https://linux.die.net/man/7/capabilities

[1] https://linux.die.net/man/8/ping

[2] https://patchwork.kernel.org/patch/10388011/

Re: Ask HN: Best Centralized Backup Solution

#50

Post by aspyct »

tomxor wrote: post #29

ZFS snapshots!... I _wanted_ to use "rsync.net" for this, they use FreeBSD with ZFS snapshots in a jail, the idea is that you get your stuff onto the account however you like over SSH (i.e rsync), then the snapshots serve two purposes: snapshots through time, and read only access incase you get 0wned. I couldn't use rsync.net though because all it's datacenters are outside of EEA, i'm currently looking into doing thi…

Note that chroot is not a security feature, it can be compromised in many ways. You probably need containers like lxc/lxd
Post reply on HN