Live data from Hacker News

Garage: Open-Source Distributed Object Storage

garagehq.deuxfleurs.fr

111–120 of 147 posts

Re: Garage: Open-Source Distributed Object Storage

#111
post #39

Whats the motivation behind project like this one? We got ceph, minio, seaweedfs ... and a dozen of others. I am genuinly curious what is the goal here?

I can only answer for Garage and not others. Garage is the result of the desired organization of the collective behind it: deuxfleurs. The model is that of people willing to establish a horizontal governance, with none being forced to do anything because it all works by consensus. The idea is to have an infrastructure serving the collective, not a self hosted thing that everyone has to maintain, not something in a data center because it has clear ecological impacts, but something in-between. Something that can be hosted on secon-hand machines, at home, but taking the low reliability of machines/electricity/residential internet into account. Some kind of cluster, but not the kind you find in the cloud where machines are supposed to be kind of always on, linked with high-bandwidth, low-latency network: quite the opposite actually.

deuxfleurs thought long and hard about the kind of infra this would translate to. The base came fast enough: some kind of storage, based on a standard (even de-facto only is good because it means it is proven), that would tolerate some nodes go down. The decision of doing a Dynamo-like thing to be accessed through S3 with eventual consistency made sense

So Garage is not "simply" a S3 storage system: it is a system to store blobs in an unreliable but still trusted coonsumer-grade network of passable machines.

Re: Garage: Open-Source Distributed Object Storage

#113

SeaweedFS is great as well. https://github.com/seaweedfs/seaweedfs

Looks awesome. Been looking for some flexible self-hosted WebDAV solutions and SeaweedFS would be an interesting choice.

depending on what you need it for nextcloud has WebDAV (clients can interact with it, and windows can mount your home folder directly, i just tried it out a couple days ago.) I've never used webdav before so i'm unsure of what other use cases there are, but the nextcloud implementation (whatever it may be) was friction-free - everything just worked.

Re: Garage: Open-Source Distributed Object Storage

#115
post #37

What I'm really missing in this space is something like this for content addressed blob storage. I feel like a lot of complexity and performance overhead could be reduced if you only store immutable blobs under their hash (e.g Blake3). Combined with a soft delete this would make all operations idempotent, blobs trivially cacheable, and all state a CRDT/monotonically mergeable/coordination free. There is stuff like IP…

But you don't really handle blobs in real life: they can't really be handled, they don't have memorable name (by design). So you need an abstractly layer on top of it. You can use zfs that will deduplicate similar blobs. You can use restic for backups that will also deduplicate similar parts of a file also in an idempotent way. And you can use git that will deduplicate files based on their hash

Re: Garage: Open-Source Distributed Object Storage

#116

Ist this formally verified by any chance ? I feel like there's space where formal designs could be expressed in TLA+ such that its easier for the community to keep track of the design.

There is formal proof for some parts of garage layout system, see https://git.deuxfleurs.fr/Deuxfleurs/garage/src/branch/main/...

Re: Garage: Open-Source Distributed Object Storage

#117

Earlier quoted context omitted.

I have done this for my purposes, but it's slow and unnecessary bloat I wish I didn't have to have.

5 hmac-sha256's per signature are slow?

Given that it's completely unnecessary for my purposes, yes. Garage doesn't support anonymous access, either.

Re: Garage: Open-Source Distributed Object Storage

#118

Earlier quoted context omitted.

It makes sense to tap into the existing ecosystem of AWS S3-compatible clients.

Plain HTTP (as in curl without any extra headers) is already an S3-compatible client. If this 'Garage' doesn't support the plain HTTP use case then it isn't S3 compatible.

Only if you are not doing auth right? If you need to auth then you need to send a request with headers.

Re: Garage: Open-Source Distributed Object Storage

#120
post #82

Earlier quoted context omitted.

Check also SeaweedFS, it has some interesting tradeoffs made, but I hear you with wanting some of the properties you're looking for.

I am using seaweed for a project right now. Some things to consider with seaweed. - It works pretty well, at least up to the 15B objects I am using it for. Running on 2 machines with about 300TB, (500 raw) storage on each. - The documentation, specifically with regards to operations like how to backup things, or different failure modes of the components can be sparse. - One example of the above is I spun up a second…

I use seaweed as well. It has some warts as well as some feature incompleteness but I think the simplicity of the project itself is a pretty nice feature. It’s grokkable mostly pretty quickly since it’s only one dev and the codebase is pretty small
Post reply on HN