Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bup – towards the perfect backup

wrouesnel.github.io

Post reply on HN
Print view View original

21–30 of 98 posts

Re: Bup – towards the perfect backup

#21

Post by lelutin »

photoGrant wrote: post #15
mappu wrote: post #14

A shoutout for attic https://attic-backup.org/ Attic is one of the new-generation hash-backup tools (like obnam, zbackup, Vembu Hive etc). It provides encrypted incremental-forever (unlike duplicity, duplicati, rsnapshot, rdiff-backup, Ahsay etc) with no server-side processing and a convenient CLI interface, and it does let you prune old backups. All other common tools seem to fail on one of the following points - In…

For someone like me who isn't very technically minded (forgive me)... Could you or someone explain why these fantastic sounding tools don't get a developed front-end? Or if they do why am I missing them? The best solution I've found is ChronoSync.

as is the case with most 100% volunteer-based free software: lack of time and/or ppl to help.

There are currently multiple interfaces available to bup, but each one has some quirks. bup's web interface is still very embryonic and would need the magic touch of some designers / integration specialists to make it fun to work with.

Re: Bup – towards the perfect backup

#22

Post by sciurus »

mappu wrote: post #14

A shoutout for attic https://attic-backup.org/ Attic is one of the new-generation hash-backup tools (like obnam, zbackup, Vembu Hive etc). It provides encrypted incremental-forever (unlike duplicity, duplicati, rsnapshot, rdiff-backup, Ahsay etc) with no server-side processing and a convenient CLI interface, and it does let you prune old backups. All other common tools seem to fail on one of the following points - In…

How does attic compare to obnam?

Re: Bup – towards the perfect backup

#23

Post by lelutin »

mynegation wrote: post #7

Deleting old backups and the lack of encryption is what stopped me from using bup.

some ppl have already started working on this and there's been activity on the mailing list lately about this topic.

however it's a dangerous feature to add in (backup tools should never screw up their storage, and this feature goes and removes things) so a lot of care is needed.

the boring answer is: it's coming, and we need a lot of help for vetting patches and testing them out.

Re: Bup – towards the perfect backup

#25

Post by mappu »

sciurus wrote: post #22
mappu wrote: post #14

A shoutout for attic https://attic-backup.org/ Attic is one of the new-generation hash-backup tools (like obnam, zbackup, Vembu Hive etc). It provides encrypted incremental-forever (unlike duplicity, duplicati, rsnapshot, rdiff-backup, Ahsay etc) with no server-side processing and a convenient CLI interface, and it does let you prune old backups. All other common tools seem to fail on one of the following points - In…

How does attic compare to obnam?

I haven't used obnam much, the feature sets are quite similar, perhaps someone else can chip in here.

My personal choice of attic over obnam was based on this performance comparison[1]

1. http://librelist.com/browser//attic/2014/1/31/attic-vs-obnam...

Re: Bup – towards the perfect backup

#26

Post by cperciva »

ars wrote: post #16
mynegation wrote: post #7

Deleting old backups and the lack of encryption is what stopped me from using bup.

And the reason they are not in bup is that those two things are the only hard things in a backup. So they did the easy parts and skipped the hard parts. Hardly a perfect backup. I'm not sure a backup with all three of: delete old backups, encryption, and upload only differences even exists. I think it might be one of those "pick any two" things, but if anyone knows of a backup with all three let me know.

Tarsnap does all of these.

Re: Bup – towards the perfect backup

#27

Post by lelutin »

Smudge wrote: post #6
rcthompson wrote: post #4

If you're considering using it, keep in mind the limitations: https://github.com/bup/bup/blob/master/README.md#things-that... The one most likely to be a showstopper seems to be: "bup currently has no way to prune old backups."

I can see how this would be theoretically possible in the same way I could see using `git filter-branch` to remove one or more commits from a code repository. But as it requires walking back up the tree to recalculate all of the commit hashes based on the new state of your files, I suspect it would be an extremely slow/expensive operation in bup's case. Someone who knows more about bup's internals can correct me if I…

true. and actually bup can't use git's tooling directly because of the different use cases it's optimized for. git won't make any use of bup's optimizations: midx (combination of multiple .idx files into a handful of bigger ones to reduce page faults during binary searches) and bloom filter.

git's tools, namely filter-branch and gc have been reported to work on limited-size bup repositories, but it very quickly eats up all ram and cpu and never finishes because of the sheer amount of objects that are usually stored in a bup repository

Re: Bup – towards the perfect backup

#28

Post by lelutin »

greensoap wrote: post #24

Given that old backups cannot be remove, isn't backuppc a better solution?

backuppc is a mature technology and can very well be trusted for collecting backups.

there are some areas in which bup is trying to innovate in order to change how backups are considered (storage size being one of those)

Re: Bup – towards the perfect backup

#29

Post by pwildani »

aidenn0 wrote: post #18
fragmede wrote: post #8

> That is a dataset which is already deduplicated via copy-on-write semantics (it was not using ZFS deduplication because you should basically never use ZFS deduplication). Can someone more experienced with ZFS say why?

"Basically never" is an overstatement, but it is true to the point of "Never unless you already know why I said 'basically never'" It boils down to the fact that ZFS maintains a mapping from hashes to LBNs. This allows write-time deduplication (as opposed to a scrubber that runs periodically and retroactively deduplicates already written blocks). This is somewhat memory intensive though. For smaller ZFS pools you can…

ZFS is also very far away from the state of the art in online dedup. For instance, http://users.soe.ucsc.edu/~avani/wildani-icde13dedup.pdf has a theoretical dedup regime that needs only 1% of the RAM for 90% of the benefit.

Re: Bup – towards the perfect backup

#30

Post by eli »

lelutin wrote: post #17
eli wrote: post #13

Is there an easy way to have the backups encrypted at rest? That's a nice feature of Duplicity. I don't have to worry about someone hacking my backup server or borrowing my USB drive having access to my data.

currently bup doesn't implement encryption (since it's a pretty hard feature to get right and we do want to finish coding other key features -- like old backup removal -- before we get to that) some ppl have reported using an encrypted storage backend like ecryptfs to store their bup repositories in. that option shouldn't be too hard to put together.

Makes sense, I'd probably prioritize pruning old backups first too. bup looks quite cool regardless, I'll be keeping an eye on it.

I like & use Duplicity, but it's a little finicky to set up right and some of the more obscure features are counterintuitive or even broken.

Post reply on HN