Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Duplicity: Encrypted bandwidth-efficient backup

duplicity.us

Post reply on HN
Print view View original

1–10 of 104 posts

Re: Duplicity: Encrypted bandwidth-efficient backup

#3

Post by mlyle »

AdmiralAsshat wrote: post #2

Brilliant name, if you think about it. If they ever decided to start doing shady shit, they'd have a perfect legal shield. No one would be able to convincingly argue in court that they were being duplicitous.

If no one can argue they're duplicitous, then it's a case of false advertising...

Re: Duplicity: Encrypted bandwidth-efficient backup

#4

Post by longwave »

I used this many, many years ago but switched to Borg[0] about five years ago. Duplicity required full backups with incremental deltas, which meant my backups ended up taking too long and using too much disk space. Borg lets you prune older backups at will, because of chunk tracking and deduplication there is no such thing as an incremental backup.

[0] https://www.borgbackup.org/

Re: Duplicity: Encrypted bandwidth-efficient backup

#5

Post by jwr »

Excellent piece of software, and relatively simple to use with gpg encryption. I've been using it for many years.

My only complaint is that, like a lot of software written in Python, it has no regard for traditional UNIX behavior (keep quiet unless you have something meaningful to say), so I have to live with cron reporting stuff like:

"/usr/lib/python2.7/dist-packages/paramiko/rsakey.py:99: DeprecationWarning: signer and verifier have been deprecated. Please use sign and verify instead. algorithm=hashes.SHA1()"

along with stuff I actually do (or might) care about.

Oh well.

Re: Duplicity: Encrypted bandwidth-efficient backup

#6

Post by 65 »

If you're using S3 to back up your files, it's easier to write a shell script with the AWS CLI. For example here's a script I wrote that I run automatically to back up my computer to S3. I have an exclude array to exclude certain folders. It's simpler than downloading software and more customizable.

# $1 # local folder

# $2 # bucket

declare -a exclude=(

  "node_modules" 

  "Applications"

  "Public"
)

args=""

for item in "${exclude[@]}";

do

  args+=" --exclude '*/$item/*' --exclude '$item/*'";
done

cmd="aws s3 sync '$1' 's3://$2$1' --include '*' $args"

eval "$cmd"

Re: Duplicity: Encrypted bandwidth-efficient backup

#8

Post by rakoo »

65 wrote: post #6

If you're using S3 to back up your files, it's easier to write a shell script with the AWS CLI. For example here's a script I wrote that I run automatically to back up my computer to S3. I have an exclude array to exclude certain folders. It's simpler than downloading software and more customizable. # $1 # local folder # $2 # bucket declare -a exclude=( "node_modules" "Applications" "Public" ) args="" for item in "${…

Your script doesn't do the same thing as duplicity. Your script mirrors the local directory with your bucket. It loses all history. Duplicity does backups (ie with history) but not just that, it does differential backups to not upload everything all the time.

Re: Duplicity: Encrypted bandwidth-efficient backup

#9

Post by _flux »

I've moved to using backup tools using content-based ids with rolling window hashes, which allows deduplicating content even between different hosts—and crucially handles moving content from one host to another efficiently—even though in other scenarios I'm guessing rdiff-algorithm can produce smaller backups.

The problem I have with duplicity and backups tools of its kind is that you still need to create a full backup again periodically, unless you want to have an ever-growing sequence of increments from the day you started doing backups.

Content-addressed backups avoid that, because all snapshots are complete (even if the backup process itself is incremental), but their content blobs are shared and eventually garbage collected when no references exist to them.

My tool of choice is kopia. Also borgbackup does similar things (though borgbackup is still unable to back up to the same repo from multiple hosts at the same time, though I haven't checked this for a while). Both do encryption, but its symmetric, so the client will have keys to opening the backups as well. If you require asymmetric encryption then these tools are not for you—though I guess this is not a technical requirement for this approach, so maybe one day a content-addressed backup tool with asymmetric encryption will appear?

Post reply on HN