Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kopia – Fast and Secure Open-Source Backup

kopia.io

Post reply on HN
Print view View original

31–40 of 80 posts

Re: Kopia – Fast and Secure Open-Source Backup

#31

Post by lucb1e »

MikusR wrote: post #28
lucb1e wrote: post #21

Earlier quoted context omitted.

Thanks! Still a bit strange to me that they started a whole new project rather than contribute patches or fork something else. The only one I'm fairly familiar with is restic. It also compiles for the major OSes and has deduplication. For compression, I think there are patches available, but alternatively they could just have contributed one. That leaves tying the command-line interface to a few buttons in a GUI. Edi…

Restic issues for adding GUI and compression are 7 years old. If if there was interest/"easy to do" for implementing them that would have been done.

What I meant is that the author of Kopia could have done that and made their life a lot easier compared to starting all the way from scratch. But I posted that before realizing that Kopia is barely a year younger than Restic.

But of course it could also just be what u/poronski said in a sibling comment. His Noodly holiness knows I make a lot of software that already exists just because I enjoy the making and having it customized. In fact I think I also started... let me `stat` that directory... yup, in 2016 I started working on (and abandoned) my own implementation of encrypted backups. Also because online storage prices were through the roof (~40x the hardware cost price with servers and bandwidth included) and I thought I could do that cheaper.

Re: Kopia – Fast and Secure Open-Source Backup

#33

Post by jbnorth »

foolinaround wrote: post #29

how would this type of software compare against Dropbox / NextCloud?

They're fundamentally different in the problems they solve. Dropbox is a cloud file storage and syncing service. NextCloud is an open source alternative to something like Dropbox in that it offers file sharing and syncing but also much more on top of that. It's really closer to something like the Google suite of personal cloud services with Google Drive, Photos, Contacts, and Calendar. Kopia is a backup solution for the files on your computer. You can use cloud file storage providers as the destination for these backups but it doesn't handle the storage of the backups itself. You have to provide that storage to it.

Re: Kopia – Fast and Secure Open-Source Backup

#34

Post by encryptluks2 »

Can I get some recent performance benchmarks vs restic? I only recently switched to restic and for the most part it is great (definitely an improvement from CrashPlan IMO), but it seems like this might actually have a few improvements to make it worth considering switching from restic.

Re: Kopia – Fast and Secure Open-Source Backup

#35

Post by lucb1e »

wmf wrote: post #20
lucb1e wrote: post #16

Earlier quoted context omitted.

How much space do you typically save by compressing these days? Given that even smallish things like documents are already compressed archives, pictures/audio/movies of course already have heavy purpose-specific compression. The main things I can still think of that are sparse on purpose are database files and disk images (not very mainstream, but also not uncommon). So like, a few gigabytes per terabyte (a few promi…

I assume compression works well for source code and other developer artifacts. Obviously you have to do dedupe, then compression, then encryption.

Source code compresses very well indeed, but my hunch is that it's peanuts. Let's see, I've got a projects directory with various projects from the past decade (all custom, there's a separate dir for downloaded repositories). I've mostly written things in Python and PHP (the JS/CSS/HTML stuff is on a server mixed with things like owncloud or SMF or so; harder to isolate).

PHP: 591 KiB, 196 files, 13'813 LOC, 1'236 comment lines.

Python: 345 KiB, 136 files, 7'760 LOC, 930 comment lines.

If someone spends 5 minutes of developer time trying to compress that to save disk space, that's already not worth it. Also in huge projects, the actual code is not going to be taking gigabytes of space. And if you mean in git history: that is, again, already compressed.

Other developer artifacts: I've got 26 GiB of project directories, this time including downloaded software and it will also include binaries (hashcat and jtr are in there, I wouldn't be surprised if there's also a medium-sized dictionary or two). Doing tar c . does not seem to add much overhead (26.5 GiB). Compressing that stream with pigz -1 (multithreaded gzip) brings it down to 17 GiB.

35% off is better than I thought! I wonder which files compress so well, hmm let me `find -type f | shuf | head -9001 | while read line; do echo "$(($(wc -c While I'm looking into this, let's also look at my "documents" directory. It's 43 GiB and compresses down to 33 GiB. Not as good, but still worth it, more than I thought! (And this compression isn't good, but probably not more than 10% worse before the compression gets impractically slow.) It might not quite get a total backup size down by a disk size (e.g. not 1T down to 500G), but it definitely allows to keep more history before having to worry about what you want to keep and what you want to toss.

Re: Kopia – Fast and Secure Open-Source Backup

#36

Post by kemonocode »

Interesting, I'd be willing to give it a try after I had to abandon Duplicati as it just seemed like a lost cause. Right now my backup setup consists on a UrBackup server my machines connect to, and a borg repository that is synced against the latest backup and then sent remotely to S3. It works, but could definitively use some streamlining...

Re: Kopia – Fast and Secure Open-Source Backup

#37

Post by ntolia »

encryptluks2 wrote: post #34

Can I get some recent performance benchmarks vs restic? I only recently switched to restic and for the most part it is great (definitely an improvement from CrashPlan IMO), but it seems like this might actually have a few improvements to make it worth considering switching from restic.

I ran some performance benchmarks not too long ago. https://blog.kasten.io/benchmarking-kopia-architecture-scale...
Post reply on HN