Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kopia – Fast and Secure Open-Source Backup

kopia.io

Post reply on HN
Print view View original

51–60 of 80 posts

Re: Kopia – Fast and Secure Open-Source Backup

#51

Post by hardwaresofton »

z77dj3kl wrote: post #50

The only thing I need (and is sorely missing from Restic) is that the metadata be kept separate from the actual data. That way I can store the data in AWS S3 Deep Glacier at a cost of nothing per year, and still do incremental backups. Currently the architecture of Restic for instance requires all data to be quickly and cheaply accessible; which makes it impossible for this. I have terabytes of data that I'd be happy…

Would you mind sharing how much you pay for S3? I assume you’ve considered options like Backblaze B2 and Wasabi?

Re: Kopia – Fast and Secure Open-Source Backup

#52

Post by jkowalski »

z77dj3kl wrote: post #50

The only thing I need (and is sorely missing from Restic) is that the metadata be kept separate from the actual data. That way I can store the data in AWS S3 Deep Glacier at a cost of nothing per year, and still do incremental backups. Currently the architecture of Restic for instance requires all data to be quickly and cheaply accessible; which makes it impossible for this. I have terabytes of data that I'd be happy…

Hi, Kopia author here.

Yes, Kopia does segregate metadata from data. Directory listings, snapshot manifests and few other minor data pieces are stored in pack files starting with "q" while the bulk of data is stored in pack files starting with "p". Indexes into both kinds of pack files are stored in filenames starting with "n".

This has some interesting properties. For example we can easily cache all metadata files locally, which provides fast directory listing, manifest listing and very fast verification without having to download actual data.

Re: Kopia – Fast and Secure Open-Source Backup

#53

Post by vladgur »

I’m curious if this or other tools can help me with the following use case: I have a Synology network attached storage.

What I want is a way for me to incrementally back up to Synology when I’m on my local network/VPN, while Synology maintains an encrypted cloud backup of my stuff. Alternatively when I’m outside of my network travelling, my laptop is backing up directly to the cloud with a way to sync that up with Synology.

In other words, I want to have a synchronized local and offline backup that works seamlessly whether I’m on my network or outside of it.

Is that a reasonable expectation? Does a tool like that exist?

Re: Kopia – Fast and Secure Open-Source Backup

#55

Post by ethanpil »

vladgur wrote: post #53

I’m curious if this or other tools can help me with the following use case: I have a Synology network attached storage. What I want is a way for me to incrementally back up to Synology when I’m on my local network/VPN, while Synology maintains an encrypted cloud backup of my stuff. Alternatively when I’m outside of my network travelling, my laptop is backing up directly to the cloud with a way to sync that up with Sy…

You can setup Zerotier on your NAS and your computer (and Phone, etc) and things will always stay connected via an internal ip no matter where you are. I have been using this setup for several years. Best part is it works with any software running on the computer or Dockers, etc on the NAS.

I bought a second (cheap,slow) NAS and installed it off-site (away from home) and I keep a full off-site mirror of my NAS.

Use any tools you want, syncthing, rsync, smb, NFS... It's all just internal IPs routed via ZeroTier.

Re: Kopia – Fast and Secure Open-Source Backup

#58

Post by shaicoleman »

This looks really cool, and seems to tick all the boxes for me:

* Multi-platform

* Encrypted

* Compressed

* Deduplication

* Supports S3 Compatible + Backblaze B2

* Performant even with millions of small files

* Supports mounting

* Good usability

Wishlist:

* Automatic exclusion from .gitignore files

* Support nested includes/excludes

Re: Kopia – Fast and Secure Open-Source Backup

#59

Post by DenseComet »

hardwaresofton wrote: post #51
z77dj3kl wrote: post #50

The only thing I need (and is sorely missing from Restic) is that the metadata be kept separate from the actual data. That way I can store the data in AWS S3 Deep Glacier at a cost of nothing per year, and still do incremental backups. Currently the architecture of Restic for instance requires all data to be quickly and cheaply accessible; which makes it impossible for this. I have terabytes of data that I'd be happy…

Would you mind sharing how much you pay for S3? I assume you’ve considered options like Backblaze B2 and Wasabi?

S3 Glacier Deep Archive is $1/TB/month. Super cheap storage costs, but retrieval costs are insane.

Re: Kopia – Fast and Secure Open-Source Backup

#60

Post by hardwaresofton »

DenseComet wrote: post #59
hardwaresofton wrote: post #51

Earlier quoted context omitted.

Would you mind sharing how much you pay for S3? I assume you’ve considered options like Backblaze B2 and Wasabi?

S3 Glacier Deep Archive is $1/TB/month. Super cheap storage costs, but retrieval costs are insane.

My question was more about just how many TBs and ingress/egress was making AWS S3 cost prohibitive -- Wasabi's sticker price is $5.99/month (so 6x glacier but ~0.2x regular S3), and I know that Hetzner will give you a storage box that is 1TB for 9.40EUR (but the kicker there is that 10TB of traffic is included which is amazing), and there are no API/operation fees when you run your own Minio (or just use FTP/all the other built-in access methods).

Network is one thing but what am I missing here? Maybe I just think $10/month is reasonable for 1TB (because I don't have enough TBs? or use remote storage enough?), and that's different from most people who are interested in this.

Post reply on HN