Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Bup – An efficient backup system based on the git packfile format

bup.github.io

Post reply on HN
Print view View original

11–20 of 40 posts

Re: Bup – An efficient backup system based on the git packfile format

#11

Post by zauberpony »

wampus wrote: post #5
mynegation wrote: post #3

I do a monthly offsite backup. I use rdiff-backup for plain text and 7z with encryption for data I want to keep private, but I am on the lookout for more efficient solutions. I looked at bup, but two things stop me from using it: no encryption, and inability to delete old backups to reclaim space. To me, attic ( https://attic-backup.org/ ) looks attractive so this is what I am going to try soon.

I've used rdiff-backup daily for years, but recently needed to store backups on machines I don't control, so I gave duplicity[1] a try. It encrypts, doesn't need to be installed on the target host and is simple to use if you're already familiar with rdiff-backup and gpg. Test restores were simple, but were all first generation, so not really a good test (but there are hints you'll want to prune backups to one month's…

You could try duply (http://duply.net/), a frontend for duplicity.

I use duply for unattended, automated backups.

Re: Bup – An efficient backup system based on the git packfile format

#12

Post by aroch »

wampus wrote: post #5
mynegation wrote: post #3

I do a monthly offsite backup. I use rdiff-backup for plain text and 7z with encryption for data I want to keep private, but I am on the lookout for more efficient solutions. I looked at bup, but two things stop me from using it: no encryption, and inability to delete old backups to reclaim space. To me, attic ( https://attic-backup.org/ ) looks attractive so this is what I am going to try soon.

I've used rdiff-backup daily for years, but recently needed to store backups on machines I don't control, so I gave duplicity[1] a try. It encrypts, doesn't need to be installed on the target host and is simple to use if you're already familiar with rdiff-backup and gpg. Test restores were simple, but were all first generation, so not really a good test (but there are hints you'll want to prune backups to one month's…

One of the downsides with duplicity is you have to provide your GPG privkey password in ENV in order for dedupe with GPG encryption to work.

Re: Bup – An efficient backup system based on the git packfile format

#13

Post by gh02t »

Since I see various different backup utilities on here from time to time, I'm curious if anybody uses Bacula (or Bareos)?

It does encryption, deduplication, scheduling, pruning etc. I use it at home and run the director and storage daemons in a jail on my FreeBSD NAS, which coordinates all of my computers together for backup regularly. I then back up the encrypted backups to Crashplan.

I tried other stuff (especially duplicity), but they just didn't work very well for me. Restoring encrypted backups with duplicity was especially slow and annoying and even using the duply frontend it was rather fickle.

Bacula has some disadvantages though... for one it is a bit complicated to get working. It's partitioned into multiple daemons, each of which handles a different task and which can be on different machines. This makes it flexible, but also makes the configuration more complicated. It also works best if you have a central backup server that can run the director and storage daemons 24/7. You can set it up to work all on one computer and just dump to a "dumb" storage drive, but it is really designed to work with a central server.

Re: Bup – An efficient backup system based on the git packfile format

#14

Post by tambourine_man »

rlpb wrote: post #4
zeograd wrote: post #2

When I had to pick a backup system, I considered Bup until I saw that there were no way to prune old backup ( https://github.com/bup/bup/blob/master/README.md#things-that... ). This is really a stopblocker for me. Obnam ( http://obnam.org ) is a similar tool but support forgetting old generations. However, it still suffers for youth problems and tends to corrupt backup repository when pruning old data on a remote ser…

I wrote ddar before I knew about bup. It works in a similar way, but uses sqlite and flat files for chunk storage, so removing old archives isn't a problem. I'm not aware of any corruption issues in ddar; I rely on sqlite for integrity. I modeled ddar after Tarsnap.

Don't know if you are aware, but http://www.synctus.com/ddar/ is currently down

Re: Bup – An efficient backup system based on the git packfile format

#15

Post by icebraining »

rlpb wrote: post #9
lucaspiller wrote: post #7

Kind of related: What do you use to backup binary files like photos? At the moment I've just got copies on old USB drives, but I'd like to put something on AWS Glacier too.

git-annex. It supports multiple external remotes, including USB drives, Glacier[1], bup and ddar[2]. You can keep files on any combination of the external remotes for redundancy and cost management. It keeps track of what is where, so you don't have to. The only catch is that you have to make sure to back up the (metadata-only) git repository itself also, and maintaining this backup on anything that is not a direct f…

+1 for git annex. For the git repo, if you don't have privacy concerns, you can just push to a private bitbucket repository. But I just had a script with:

  git clone /my/repo /tmp/backup-git-annex
  tar cvf /tmp/backup-git-annex.tar /tmp/backup-git-annex
That way, you're left with a single tar file to backup.

Re: Bup – An efficient backup system based on the git packfile format

#16

Post by rlpb »

tambourine_man wrote: post #14
rlpb wrote: post #4

Earlier quoted context omitted.

I wrote ddar before I knew about bup. It works in a similar way, but uses sqlite and flat files for chunk storage, so removing old archives isn't a problem. I'm not aware of any corruption issues in ddar; I rely on sqlite for integrity. I modeled ddar after Tarsnap.

Don't know if you are aware, but http://www.synctus.com/ddar/ is currently down

I'm aware, thanks. https://github.com/basak/ddar/wiki contains the content, thanks to a kind contributor. I should probably deprecate the old URL and remove references to it.

Re: Bup – An efficient backup system based on the git packfile format

#18

Post by rcthompson »

rlpb wrote: post #9
lucaspiller wrote: post #7

Kind of related: What do you use to backup binary files like photos? At the moment I've just got copies on old USB drives, but I'd like to put something on AWS Glacier too.

git-annex. It supports multiple external remotes, including USB drives, Glacier[1], bup and ddar[2]. You can keep files on any combination of the external remotes for redundancy and cost management. It keeps track of what is where, so you don't have to. The only catch is that you have to make sure to back up the (metadata-only) git repository itself also, and maintaining this backup on anything that is not a direct f…

So git-annex now has support for efficiently deduplicating backends? Might be time for me to give it another look.

Re: Bup – An efficient backup system based on the git packfile format

#19

Post by rlpb »

rcthompson wrote: post #18
rlpb wrote: post #9

Earlier quoted context omitted.

git-annex. It supports multiple external remotes, including USB drives, Glacier[1], bup and ddar[2]. You can keep files on any combination of the external remotes for redundancy and cost management. It keeps track of what is where, so you don't have to. The only catch is that you have to make sure to back up the (metadata-only) git repository itself also, and maintaining this backup on anything that is not a direct f…

So git-annex now has support for efficiently deduplicating backends? Might be time for me to give it another look.

For "local" storage as supported by bup and ddar, yes. Generally for external remotes: not yet, AFAIK. http://git-annex.branchable.com/design/assistant/deltas/ has some design thoughts, so I think it is planned.

Re: Bup – An efficient backup system based on the git packfile format

#20

Post by sliken »

gh02t wrote: post #13

Since I see various different backup utilities on here from time to time, I'm curious if anybody uses Bacula (or Bareos)? It does encryption, deduplication, scheduling, pruning etc. I use it at home and run the director and storage daemons in a jail on my FreeBSD NAS, which coordinates all of my computers together for backup regularly. I then back up the encrypted backups to Crashplan. I tried other stuff (especially…

Last I checked bacula doesn't do deduplication in any normal sense of the word. It allows you to run a base job and then back up diffs, that's a far cry from normal dedupe.
Post reply on HN