Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kopia: Fast and secure open-source backup software

kopia.io

Post reply on HN
Print view View original

121–130 of 138 posts

Re: Kopia: Fast and secure open-source backup software

#121

Post by e12e »

xgbi wrote: post #63

We are trying to use it for large backups of a production item, and it has not been a complete smooth ride all along. We have many files (millions) and lots of churn over ~80Tb total. Kopia has exhibited some issues: - takes about 120GB (!) of ram to perform regular maintenance & takes about 5hrs to do so. There are ideas floating around to cherry pick the large inefficiencies in the GC code but it’s yet to be worked…

> If I could use zfs cheaply on azure via s3

Can't you?

Either with basically:

  zfs send|rclone rcat
Or more directly via s3 tooling?

https://github.com/presslabs/z3

https://github.com/michabbs/rclone-sync-zfs

https://github.com/ACiDGRiM/RcloneZFSBackup

Re: Kopia: Fast and secure open-source backup software

#122

Post by shaicoleman »

aborsy wrote: post #119

How does it compare with Restic, which is very similar, and written in GO? One feature that I wished these tools would have provided is support for public key encryption. The secret key then need not be exposed.

bupstash supports it, however I didn't try it out

https://github.com/andrewchambers/bupstash/blob/master/doc/g...

Re: Kopia: Fast and secure open-source backup software

#123

Post by bscphil »

tux2bsd wrote: post #104
AnonHP wrote: post #98

Borg 2 has been in development for nearly a year and a half [1] and may probably be released early next year, i.e., early 2024 (just a guess, seeing that even RC1 is not yet released and seems to have a lot of work to be done). Does anyone know how Borg 1.x and 2 would compare to Kopia? [1]: https://github.com/borgbackup/borg/issues/6602

I tried borg and liked it but I couldn't see how to purge 'file.big' out of prior backups. i.e. one you knew for sure you didn't need any more and would achieve disk space recovery.

You can do it, although the ergonomics are not terrific: https://borgbackup.readthedocs.io/en/stable/usage/recreate.h... (see the examples).

Re: Kopia: Fast and secure open-source backup software

#124

Post by bscphil »

justin_oaks wrote: post #29

I use Restic for personal backups and at work, but I thought I'd try Kopia to see if it'd be a good fit for my less techie relatives. This was about a year ago. The UI didn't seem like a good fit for those who are less technical. I don't remember the specifics. Does anyone have recommendations for backup services for the average user?

I used to use Restic many years ago, when it wasn't clear what the future direction of Borg (a.k.a Attic) would be since development had mostly stalled. At the time it was missing some features and Borg managed to fix its issues quickly enough that I ended up using that.

These days, from the minimal time I've spent looking at this, it seems that Borg and Restic offer basically the same feature set with similar performance. I'm curious if you (or anyone else) considered Borg and what set Restic apart for you.

Ditto for Kopia, I guess. I've never heard of it before.

Re: Kopia: Fast and secure open-source backup software

#125

Post by FieryTransition »

JoshuaEN wrote: post #83
FieryTransition wrote: post #75

Used it for a while, recently tried to restore some things and it failed, taking a really long time to restore some snapshots compared to other things I've tried. Switched to restic instead. Really like what kopia is but I'll wait a few more years before considering it for something, but right now I'm happy with restic. Too bad no one besides kopia does ecc, which is the reason I switched, but when I checked out why…

This has been my experience too with Kopia. I tried to restore a ~200 GB file (stored remotely on a Hetzner Storage Box), and it failed (or at least did not finish after being left for ~20 hours; there was also no progress indicator or status I could find in the UI). I also tried to restore a folder with about ~32 GB of data in it, and that also failed (the UI did report an error, but I don't recall it being useful).…

yeah, I had some weirdness with the UI and disconnects as well. My takeaway from trying it was that I wouldn't want to use it for something if I need peace of mind for my data.

Re: Kopia: Fast and secure open-source backup software

#126

Post by MenhirMike »

donmcronald wrote: post #109
MenhirMike wrote: post #46

CTRL+F "tape" "lto" - 0 results. I know that Tape Backups are not hip and sexy, but CloudNordic showed us just last month why they still matter even in 2023 and beyond, so you'd definitely want to look at an additional solution for your large servers, with a proper rotation/retention strategy (e.g., GFS). You _need_ offline backups, if you think you don't, you just got lucky for now - or have data that can be recreat…

> backing up a running Postgresql server through a file system snapshot may not be enough - there's an entire section in the documentation about backup options https://www.postgresql.org/docs/16/backup-file.html > An alternative file-system backup approach is to make a “consistent snapshot” of the data directory, if the file system supports that functionality (and you are willing to trust that it is implemented corre…

Yeah, I think I really didn't like the "The server thinks it's crashed and will run a recovery" part, but they do go out of their way to call out "It's fine, just make sure the WAL is also backed up".

Can't even claim that this is a recent addition, since it's documented like that since Postgresql 8, which was released in 2005.

Re: Kopia: Fast and secure open-source backup software

#127

Post by xgbi »

djbusby wrote: post #87
xgbi wrote: post #63

We are trying to use it for large backups of a production item, and it has not been a complete smooth ride all along. We have many files (millions) and lots of churn over ~80Tb total. Kopia has exhibited some issues: - takes about 120GB (!) of ram to perform regular maintenance & takes about 5hrs to do so. There are ideas floating around to cherry pick the large inefficiencies in the GC code but it’s yet to be worked…

If you're looking for ZFS backed why not rsync.net? Am I dumb for just doing some rclone+rsync.net?

I need to store on Azure for DRP reasons: we would DRP to azure and need the bandwidth.

Also, storing the ZFS snapshots on Blob storage would still require us to retrieve the entirety of the 80TB before being able to use it. I need native ZFS at Blobstore-competitive prices

Re: Kopia: Fast and secure open-source backup software

#128

Post by Linux-Fan »

dsissitka wrote: post #73
Linux-Fan wrote: post #67

Earlier quoted context omitted.

Bupstash ( https://bupstash.io/ ) beats Borg and Kopia in my tests (see https://masysma.net/37/backup_tests_borg_bupstash_kopia.xhtm... ). It is a modern take very close to what Borg offers regarding the feature set but has a significantly better performance (in terms of resource use for running tasks, the backups were slightly larger than Borg's in my tests).

> Borg ... multiple hosts backup to same target ... Yes It might be worth adding an asterisk there. https://borgbackup.readthedocs.io/en/stable/faq.html#can-i-b...

Indeed, thank you for pointing this out. I have added a footnote to the page accordingly.

Re: Kopia: Fast and secure open-source backup software

#129

Post by pdimitar »

shaicoleman wrote: post #122
aborsy wrote: post #119

How does it compare with Restic, which is very similar, and written in GO? One feature that I wished these tools would have provided is support for public key encryption. The secret key then need not be exposed.

bupstash supports it, however I didn't try it out https://github.com/andrewchambers/bupstash/blob/master/doc/g...

I like bupstash but it doesn't compress well. Admittedly I haven't tried it in several months but Borg absolutely eats it for breakfast with zstd compression.

Re: Kopia: Fast and secure open-source backup software

#130

Post by pdimitar »

Linux-Fan wrote: post #67
monlockandkey wrote: post #36

Can someone please help decide what is the "best" backup software? - Restic ( https://restic.net/ ) - Borg backup ( https://www.borgbackup.org/ ) - Duplicati ( https://www.duplicati.com/ ) - Kopia ( https://kopia.io/ ) - Duplicay ( https://duplicacy.com/ ) - Duplicity ( https://duplicity.us/ )

Bupstash ( https://bupstash.io/ ) beats Borg and Kopia in my tests (see https://masysma.net/37/backup_tests_borg_bupstash_kopia.xhtm... ). It is a modern take very close to what Borg offers regarding the feature set but has a significantly better performance (in terms of resource use for running tasks, the backups were slightly larger than Borg's in my tests).

Last I tried it, it didn't compress well. Did it improve? Does it have zstd compression?
Post reply on HN