Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Duplicity: Encrypted bandwidth-efficient backup

duplicity.us

Post reply on HN
Print view View original

101–104 of 104 posts

Re: Duplicity: Encrypted bandwidth-efficient backup

#101

Post by johnchristopher »

jszymborski wrote: post #56
johnchristopher wrote: post #39

Earlier quoted context omitted.

> Remember folks, test your backups. Since you mention it, I am seizing the opportunity to ask: how should borg backup be tested ? Can it be automated ?

It's actually pretty simple using the check command [0]! borg check --verify-data REPOSITORY_OR_ARCHIVE You can add that to a cron job. Alternatively, I think the Vorta GUI also has a way to easily schedule it[1]. I'll add that one thing I like to do once in a blue-moon is to spin-up a VM and try to recover a few random files. While the check command checks that the data is there and theoretically recoverable, nothin…

> It's actually pretty simple using the check command [0]!

> borg check --verify-data REPOSITORY_OR_ARCHIVE

Thanks ! I thought there were some more convoluted process but I couldn't picture out anything except extracting the whole archives and check up by hand.

Re: Duplicity: Encrypted bandwidth-efficient backup

#102

Post by nijave »

nh2 wrote: post #67
nijave wrote: post #48

Earlier quoted context omitted.

>borgbackup is still unable to back up to the same repo from multiple hosts at the same time, Basically still an issue. The machine takes an exclusive lock and it also adds override since each machine has to update it's local data cache (or whatever it's called) because they're constantly getting out of sync when another machine backs up bupstash looks promising as a close-to-but-more-performant borg alternative but…

What is alpha quality about bupstash? It has less features than Kopia, but what's there looks like high-quality to me. (I'm also using it to back up 150 TB (300 million files), on which all other dedup programs run out of memory.)

I guess the docs call it "beta" so maybe that's more accurate. Last I looked a couple months ago there's still a fair bit of corruption issues cropping up

Re: Duplicity: Encrypted bandwidth-efficient backup

#103

Post by pdimitar »

stavros wrote: post #79
pdimitar wrote: post #72

Earlier quoted context omitted.

My problem with Restic was that it did not recognize sub-second timestamps of files. I made test scripts that tested it (and were creating files and directories in a hypothetical backup source, and were also changing the files) but then Restic insisted nothing was changed because the changes were happening too fast. I modified the scripts to do `sleep 1` between each change but it left a sour taste and I never gave R…

I'm going to say that was a bit of a niche usage :P

I tried Restic again but, its repo size is 2x of that of Borg which allows you to fine-tune compression, and Restic doesn't.

So I'll keep an eye on Rustic instead (it is much faster on some hot paths + allows you to specify base path of the backup; long story but I need that feature a lot because I also do copies of my stuff to network disks and when you backup from there you want to rewrite the path inside the backup snapshot).

Rustic compresses equivalently to Borg which is not a surprise because both use zstd on the same compression level.

Re: Duplicity: Encrypted bandwidth-efficient backup

#104

Post by Matumio »

GTP wrote: post #95
_flux wrote: post #9

I've moved to using backup tools using content-based ids with rolling window hashes, which allows deduplicating content even between different hosts—and crucially handles moving content from one host to another efficiently—even though in other scenarios I'm guessing rdiff-algorithm can produce smaller backups. The problem I have with duplicity and backups tools of its kind is that you still need to create a full back…

> their content blobs are shared Doesn't this increase the chance of data loss? If a blob gets corrupted, then all the backups referencing that blob will have the same corrupted file(s). This is similar to having a corrupted index in an incremental backup chain (or maybe in this case you would lose everything?), but in the case of incremental backups the risk is mitigated by periodically performing full backups. Also…

Most backup tools allow you to verify a random subset, say 0.1% of the backup. If you do that together with every backup you would eventually notice.
Post reply on HN