Bup: Efficient file backup system based on the git packfile format

Bup is lovely. I used it to back up my huge home folder and only switched away to rdiff-backup because (at the time) there was no support for deleting old revisions.

Is there any support for that? (Of course, for a large enough hard drive, it's not much of a problem...)

Re: Bup: Efficient file backup system based on the git packfile format

#5

Post by jitl » Sat, Feb 15, 2014, 10:27 PM UTC

gcr wrote: post #4

Bup is lovely. I used it to back up my huge home folder and only switched away to rdiff-backup because (at the time) there was no support for deleting old revisions. Is there any support for that? (Of course, for a large enough hard drive, it's not much of a problem...)

Looking at the README under "Stuff that is stupid":

  > bup currently has no way to prune old backups

Thanks for the rdiff-backup shout-out. I'm looking for a nice way to do system backups to my NAS of large VM images without having to install Crashplan. Bup and rdiff-backup both look pretty good.

Re: Bup: Efficient file backup system based on the git packfile format

#6

Post by staticshock » Sat, Feb 15, 2014, 10:30 PM UTC

mattdeboard wrote: post #3

I'm assuming you're not the author. But just in case the author wanders by: How did you decide which parts to write in C?

The author is https://news.ycombinator.com/user?id=apenwarr, though it looks like he isn't the most active maintainer nowadays.

Re: Bup: Efficient file backup system based on the git packfile format

#7

Post by nathancahill » Sat, Feb 15, 2014, 10:31 PM UTC

mattdeboard wrote: post #3

I'm assuming you're not the author. But just in case the author wanders by: How did you decide which parts to write in C?

Not the author either, but I've made that decision based on profiling the code. It's generally easy to see which functions are slowing down performance and can be refactored to a more performant language.

Re: Bup: Efficient file backup system based on the git packfile format

#8

Post by gesman » Sat, Feb 15, 2014, 10:53 PM UTC

This is an outstanding project with great potential.

Killer for many commercial, overpriced services.

Re: Bup: Efficient file backup system based on the git packfile format

#9

Post by apenwarr » Sat, Feb 15, 2014, 10:56 PM UTC

mattdeboard wrote: post #3

I'm assuming you're not the author. But just in case the author wanders by: How did you decide which parts to write in C?

It's not too hard actually. A line of python is roughly 80x slower than a line of C (no exaggeration). But a typical line of python does a lot more than a typical line of C. So things you can do with a "loose" loop (like once per 64k block) is usually ok in python. Things you have to do with a "tight" loop (like once per byte) need to be in C.

I once did a presentation about python performance optimization lessons from bup: http://lanyrd.com/2011/pycodeconf/sghxk/

And it's true, I'm not the most active maintainer anymore. The people who took over seem to be doing a pretty good job though.

Re: Bup: Efficient file backup system based on the git packfile format

#10

Post by natch » Sat, Feb 15, 2014, 10:57 PM UTC

How do people who would use this kind of thing manage to have remote servers with terabytes of available disk space on them?

Anything is possible with money, of course, but how is this anything other than really expensive?

For example AWS S3 would be $235/month (that's $2,820/year!) for 3TB not even including any data-out transfer charges. Sure there are others that are cheaper but only marginally so.

Is this really what people are doing? Makes the commercial services sound really cheap.