Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

github.com

Post reply on HN
Print view View original

31–40 of 72 posts

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#31

Post by Mister_Snuggles »

For about the last year or so I've been looking for an online backup system with the following requirements:

- Off-site storage, preferably not costing too much.

- Option for on-site storage (e.g., to store a backup "in the cloud" and on my NAS)

- Keeps version history, with the associated goodies (purging old backups, etc)

- Able to run on FreeBSD and Linux, with Windows and MacOS being nice to have but not required.

- Able to back up multiple machines to one account.

I strongly suspect that my solution will involve two separate things - one to actually do the backups and another for the storage.

So far, not having looked at Duplicacy, I'm leaning strongly towards attic/borg with rsync.net for off-site storage. At first glance, Duplicacy looks like it will meet my requirements so I will have to give it a closer look before I pick a solution.

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#32

Post by acrosync »

dom0 wrote: post #17

The verdict of the "open source competition" in Duplicacy's README is not entirely accurate. Exclusive locking in the sync'd approach is just the easiest implementation, not the sole possibility. I can't speak for other tools, since I do not know their internals well enough, but I can say about Borg ( http://www.borgbackup.org/ ) that there is no inherent issue in running the important parts of making backups (i.e. u…

I might be wrong but I want to hear more from you if you're a Borg developer. My understanding is that you may be able to have multiple clients uploading chunks at the same time, but you won't be able to exploit cross-client deduplication if different clients have a similar set of files (OS files or a large code base for instance). Moreover, if your implementation require locks then it would be very hard to extend to cloud services.

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#33

Post by lfowles »

Mister_Snuggles wrote: post #31

For about the last year or so I've been looking for an online backup system with the following requirements: - Off-site storage, preferably not costing too much. - Option for on-site storage (e.g., to store a backup "in the cloud" and on my NAS) - Keeps version history, with the associated goodies (purging old backups, etc) - Able to run on FreeBSD and Linux, with Windows and MacOS being nice to have but not required…

Have you looked at crashplan? Although I can't say I've tried running it on FreeBSD...

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#34

Post by wmf »

X86BSD wrote: post #25

The website makes it sound just like Tarsnap. Am I wrong? Is there some compelling feature I am missing that would make me want to switch from Tarsnap?

AFAIK Tarsnap backups go through Tarsnap's server but every other backup tool seems to not have that requirement.

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#36

Post by avar »

Mister_Snuggles wrote: post #31

For about the last year or so I've been looking for an online backup system with the following requirements: - Off-site storage, preferably not costing too much. - Option for on-site storage (e.g., to store a backup "in the cloud" and on my NAS) - Keeps version history, with the associated goodies (purging old backups, etc) - Able to run on FreeBSD and Linux, with Windows and MacOS being nice to have but not required…

I use git-annex which satisfies almost all of those requirements. I don't back up whole machines (why backup the OS when I can reinstall faster), but it otherwise has all those things.

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#37

Post by [deleted] »

koolba wrote: post #8
acrosync wrote: post #2

Developer here. Duplicacy is built on the concept of Lock-Free Deduplication ( https://github.com/gilbertchen/duplicacy/blob/master/DESIGN.... ), which allows it to backup multiple computers to the same storage without using any locks. Currently it supports local or networked drives, SFTP servers, Amazon S3, Backblaze B2, Microsoft Azure, Google Cloud Storage, Google Drive, OneDrive, Dropbox, and Hubic. I recently re…

From the link to the license: > Fair Source has the power to promote diversity within the developer community . To date, contributing to open source has been an expensive proposition for developers. You have to have a stable income and a lot of extra time to work on side projects for free, which means talented developers from underprivileged backgrounds often aren’t able to contribute. Fair Source allows developers t…

[deleted]

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#38

Post by vertex-four »

kefka wrote: post #27

Reported to Github, as Commercial software masquerading as various open free license projects (MIT, GPL, BSD, etc.). Also, intentional namespace pollution with existing backup tool, which IS gpl'ed. Not cool. Not cool at all. ____________________________________ (response, since I'm submitting 'too fast'... ): Github has commercial repos, and private repos. It's pretty simple, really. If you want the free options on…

Wait, how's it masquerading as what?

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#39

Post by acrosync »

bascule wrote: post #23

Some claims: "It is the only cloud backup tool that allows multiple computers to back up to the same storage simultaneously without using any locks (thus readily amenable to various cloud storage services)" "What is novel about lock-free deduplication is the absence of a centralized indexing database for tracking all existing chunks and for determining which chunks are not needed any more. Instead, to check if a chun…

Naming chunks by their hashes is not a new idea, but this technique along does not give you a practical backup tool. The deletion of unreferenced chunks becomes a hard problem, and the center piece of lock-free deduplication is the two-step fossil collection algorithm that solves this hard problem.

Re: Duplicacy: Lock-free deduplication cloud backup tool, with “fair source” license

#40

Post by Mister_Snuggles »

avar wrote: post #36
Mister_Snuggles wrote: post #31

For about the last year or so I've been looking for an online backup system with the following requirements: - Off-site storage, preferably not costing too much. - Option for on-site storage (e.g., to store a backup "in the cloud" and on my NAS) - Keeps version history, with the associated goodies (purging old backups, etc) - Able to run on FreeBSD and Linux, with Windows and MacOS being nice to have but not required…

I use git-annex which satisfies almost all of those requirements. I don't back up whole machines (why backup the OS when I can reinstall faster), but it otherwise has all those things.

I wasn't clear when I said "multiple machines" - I'm definitely not backing up whole machines!

It would be a case of multiple machines each backing up their /etc, /home, /var, etc to one place.

Post reply on HN