Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kopia – Fast and Secure Open-Source Backup

kopia.io

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Kopia – Fast and Secure Open-Source Backup

#71

Post by luckman212 »

ethanpil wrote: post #55
vladgur wrote: post #53

I’m curious if this or other tools can help me with the following use case: I have a Synology network attached storage. What I want is a way for me to incrementally back up to Synology when I’m on my local network/VPN, while Synology maintains an encrypted cloud backup of my stuff. Alternatively when I’m outside of my network travelling, my laptop is backing up directly to the cloud with a way to sync that up with Sy…

You can setup Zerotier on your NAS and your computer (and Phone, etc) and things will always stay connected via an internal ip no matter where you are. I have been using this setup for several years. Best part is it works with any software running on the computer or Dockers, etc on the NAS. I bought a second (cheap,slow) NAS and installed it off-site (away from home) and I keep a full off-site mirror of my NAS. Use a…

ZeroTier is incredibly good. I have a Synology too and without ZT it would be a lot less useful.

Re: Kopia – Fast and Secure Open-Source Backup

#72

Post by z77dj3kl »

hardwaresofton wrote: post #70
z77dj3kl wrote: post #68

Earlier quoted context omitted.

Suppose I have ~4 TB of data. If I dump it into Deep Glacier, it'll be ~$50/year (free ingress), if I ever need to retrieve the data, it's like $370. Normal S3 would be ~$1.1k/yr, or around half of that for the infrequent access tier, both of which are way too expensive.

Thanks for sharing! This is why I asked... Wasabi prices 4TB of data, with 100% of it downloaded every month @ $287/year according to their price calculator. Backblaze B2's calculator is a little more sophisticated, and putting in the numbers for an absolutely pathological usecase where you start with 4TB, download, delete and upload that same amount every month puts you at $720/year. A much less pathological use cas…

This wouldn't be the primary backup, but Deep Glacier is just such a good deal that: I'd be happy to pay the $50 per year for a call option on my data, it'd make me sleep better at night!

Part of my calculus is that I have quite strong confidence in AWS in terms of business continuity and reliability/availability. If I dump my files on AWS, I have high confidence in the files (and AWS) being around in 10 years and retrievable for roughly the same price (or at least no more).

Hetzner would have much lower durability. I'm a bit suss on Backblaze, though I do trust them to be more durable than my self-managed disks (and uncorrelated to my failures). I don't know much about Wasabi; but it's not a good sign for me that their landing page touts their latest funding round at the top: seems young and you never know if the price is subsidized with VC money (and won't be in n years) or similar.

> Also, what happens if you have to retrieve data twice from Glacier?

The killer is the egress. I'd just buy a new set of disks and download it straight there.

Re: Kopia – Fast and Secure Open-Source Backup

#73

Post by z77dj3kl »

jkowalski wrote: post #52
z77dj3kl wrote: post #50

The only thing I need (and is sorely missing from Restic) is that the metadata be kept separate from the actual data. That way I can store the data in AWS S3 Deep Glacier at a cost of nothing per year, and still do incremental backups. Currently the architecture of Restic for instance requires all data to be quickly and cheaply accessible; which makes it impossible for this. I have terabytes of data that I'd be happy…

Hi, Kopia author here. Yes, Kopia does segregate metadata from data. Directory listings, snapshot manifests and few other minor data pieces are stored in pack files starting with "q" while the bulk of data is stored in pack files starting with "p". Indexes into both kinds of pack files are stored in filenames starting with "n". This has some interesting properties. For example we can easily cache all metadata files l…

Thanks! I'll have to investigate Kopia properly.

Re: Kopia – Fast and Secure Open-Source Backup

#74

Post by quantumofalpha »

hardwaresofton wrote: post #51
z77dj3kl wrote: post #50

The only thing I need (and is sorely missing from Restic) is that the metadata be kept separate from the actual data. That way I can store the data in AWS S3 Deep Glacier at a cost of nothing per year, and still do incremental backups. Currently the architecture of Restic for instance requires all data to be quickly and cheaply accessible; which makes it impossible for this. I have terabytes of data that I'd be happy…

Would you mind sharing how much you pay for S3? I assume you’ve considered options like Backblaze B2 and Wasabi?

Google Workspace is still around $10/month for unlimited. 50TB here and counting, uploaded a few more TBs just this week. Incredible value proposition! Only some limits on traffic, 750GB/day ingress. Works very well with rclone.

Legally that price is supposed to be for 1TB storage but the quota is still not being technically enforced, at least for me with a grandfathered gsuite account. Not sure about new accounts, it seems something may have changed couple of weeks ago with new ToS.

Re: Kopia – Fast and Secure Open-Source Backup

#75

Post by hardwaresofton »

quantumofalpha wrote: post #74
hardwaresofton wrote: post #51

Earlier quoted context omitted.

Would you mind sharing how much you pay for S3? I assume you’ve considered options like Backblaze B2 and Wasabi?

Google Workspace is still around $10/month for unlimited. 50TB here and counting, uploaded a few more TBs just this week. Incredible value proposition! Only some limits on traffic, 750GB/day ingress. Works very well with rclone. Legally that price is supposed to be for 1TB storage but the quota is still not being technically enforced, at least for me with a grandfathered gsuite account. Not sure about new accounts, i…

Wow... this is insane, nothing can compete with that, I'm pretty sure! 750GB/day ingress is huge.

Surely they've closed this "loophole" on new accounts.

Re: Kopia – Fast and Secure Open-Source Backup

#76

Post by quantumofalpha »

hardwaresofton wrote: post #75
quantumofalpha wrote: post #74

Earlier quoted context omitted.

Google Workspace is still around $10/month for unlimited. 50TB here and counting, uploaded a few more TBs just this week. Incredible value proposition! Only some limits on traffic, 750GB/day ingress. Works very well with rclone. Legally that price is supposed to be for 1TB storage but the quota is still not being technically enforced, at least for me with a grandfathered gsuite account. Not sure about new accounts, i…

Wow... this is insane, nothing can compete with that, I'm pretty sure! 750GB/day ingress is huge. Surely they've closed this "loophole" on new accounts.

It's been an open secret for many years that they don't enforce quotas. It's not profitable but google has really deep pockets and can afford to not care. Not sure if they really closed it this time - like I said, I'm still able to upload multiple extra terabytes onto my supposedly 1 TB account even now. So it doesn't seem like they started enforcing quotas to me.

Look/ask around in r/DataHoarder for recent experiences of other people, they also discuss other storage services in general a lot.

Re: Kopia – Fast and Secure Open-Source Backup

#77

Post by mekster »

z77dj3kl wrote: post #72
hardwaresofton wrote: post #70

Earlier quoted context omitted.

Thanks for sharing! This is why I asked... Wasabi prices 4TB of data, with 100% of it downloaded every month @ $287/year according to their price calculator. Backblaze B2's calculator is a little more sophisticated, and putting in the numbers for an absolutely pathological usecase where you start with 4TB, download, delete and upload that same amount every month puts you at $720/year. A much less pathological use cas…

This wouldn't be the primary backup, but Deep Glacier is just such a good deal that: I'd be happy to pay the $50 per year for a call option on my data, it'd make me sleep better at night! Part of my calculus is that I have quite strong confidence in AWS in terms of business continuity and reliability/availability. If I dump my files on AWS, I have high confidence in the files (and AWS) being around in 10 years and re…

I suppose you can't check for backup data integrity inside Glacier.

Re: Kopia – Fast and Secure Open-Source Backup

#78

Post by hardwaresofton »

quantumofalpha wrote: post #76
hardwaresofton wrote: post #75

Earlier quoted context omitted.

Wow... this is insane, nothing can compete with that, I'm pretty sure! 750GB/day ingress is huge. Surely they've closed this "loophole" on new accounts.

It's been an open secret for many years that they don't enforce quotas. It's not profitable but google has really deep pockets and can afford to not care. Not sure if they really closed it this time - like I said, I'm still able to upload multiple extra terabytes onto my supposedly 1 TB account even now. So it doesn't seem like they started enforcing quotas to me. Look/ask around in r/DataHoarder for recent experienc…

Oh yeah I know, I just didn't think that people wre doing that much with it! I definitely lurk r/datahoarder, absolutely love seeing people that are so excited about storage (and haven't been found by mainstream reddit for the most part yet). r/zfs is also pretty good for nerdy drive stuff from time to time.

Re: Kopia – Fast and Secure Open-Source Backup

#79

Post by nijave »

tut-urut-utut wrote: post #9

Looks interesting. I always liked the borg concept, but never actually used it for backups because it lacks windows support and a nice gui. Will give it a try.

It doesn't have native Windows support but it works pretty well in Cygwin. I attempted to create a minimal Windows/Cygwin setup for it to run in but admittedly it's in a bit of decay at the current moment https://github.com/nijave/borg-windows-package
Post reply on HN