91–100 of 147 posts
What is the difference between a "distributed object storage" and a file system?
It’s an S3 api compatible object store that supports distributed storage across different servers. Object store = store blobs of bytes. Usually by bucket + key accessible over HTTP. No POSIX expectation. Distributed = works spread across multiple servers in different locations.
Files
by bucket
Directories
key accessible
File names
over HTTP
Web server
What I'm really missing in this space is something like this for content addressed blob storage. I feel like a lot of complexity and performance overhead could be reduced if you only store immutable blobs under their hash (e.g Blake3). Combined with a soft delete this would make all operations idempotent, blobs trivially cacheable, and all state a CRDT/monotonically mergeable/coordination free. There is stuff like IP…
Check also SeaweedFS, it has some interesting tradeoffs made, but I hear you with wanting some of the properties you're looking for.
- It works pretty well, at least up to the 15B objects I am using it for. Running on 2 machines with about 300TB, (500 raw) storage on each.
- The documentation, specifically with regards to operations like how to backup things, or different failure modes of the components can be sparse.
- One example of the above is I spun up a second filer instance (which is supposed to sync automatically) which caused the master server to emit an error while it was syncing. The only way to know if it was working was watching the new filers storage slowly grow.
- Seaweed has a pretty high bus factor, though the dev is pretty responsive and seems to accept PRs at a steady rate.
What I'm really missing in this space is something like this for content addressed blob storage. I feel like a lot of complexity and performance overhead could be reduced if you only store immutable blobs under their hash (e.g Blake3). Combined with a soft delete this would make all operations idempotent, blobs trivially cacheable, and all state a CRDT/monotonically mergeable/coordination free. There is stuff like IP…
Take a look at https://github.com/n0-computer/iroh Open source project written in Rust that uses BLAKE3 (and QUIC, which you mentioned in another comment)
E.g. it tries to solve the "mutability problem" (having human readable identifiers point to changing blobs); there are blobs and collections and documents; there is a whole resolver system with their ticket stuff
All of these things are interesting problems, that I'd definitely like to see solved some day, but I'd be more than happy with an "S3 for blobs" :D.
Whats the motivation behind project like this one? We got ceph, minio, seaweedfs ... and a dozen of others. I am genuinly curious what is the goal here?
It is an object storage system and more..
What I'm really missing in this space is something like this for content addressed blob storage. I feel like a lot of complexity and performance overhead could be reduced if you only store immutable blobs under their hash (e.g Blake3). Combined with a soft delete this would make all operations idempotent, blobs trivially cacheable, and all state a CRDT/monotonically mergeable/coordination free. There is stuff like IP…
I have used Garage for a long time. It's great, but the AWS sigv4 protocol for accessing it is just frustrating. Why can't I just send my API key as a header? I don't need the full AWS SDK to get and put files, and the AWS sigv4 is a ton of extra complexity to add to my projects. I don't care about the "security benefits" of AWS sigv4. I hope the authors consider a different authentication scheme so I can recommend G…
https://github.com/Peergos/Peergos/blob/master/src/peergos/s...
Earlier quoted context omitted.
quick question for advice - we have been evaluating minio for a in-house deployed storage for ML data. this is financial data which we have to comply on a crap ton of regulations. so we wanted lots of compliance features - like access logs, access approvals, short lived (time bound) accesses, etc etc. how would you compare garage vs minio on that front ?
You will probably put a proxy in front of it, so do your audit logging there (nginx ingress mirror mode works pretty good for that)
Plus, in the spirit of open source, it's very likely that if one person has this need then others have this need, too, and thus the whole ecosystem grows versus everyone having one more point of failure in the HTTP traversal
Earlier quoted context omitted.
You will probably put a proxy in front of it, so do your audit logging there (nginx ingress mirror mode works pretty good for that)
As a competing theory, since both Minio and Garage are open source, if it were my stack I'd patch them to log with the granularity one wished since in my mental model the system of record will always have more information than a simple HTTP proxy in front of them Plus, in the spirit of open source, it's very likely that if one person has this need then others have this need, too, and thus the whole ecosystem grows ve…
What I'm really missing in this space is something like this for content addressed blob storage. I feel like a lot of complexity and performance overhead could be reduced if you only store immutable blobs under their hash (e.g Blake3). Combined with a soft delete this would make all operations idempotent, blobs trivially cacheable, and all state a CRDT/monotonically mergeable/coordination free. There is stuff like IP…
The RADOS K/V store is pretty close. Ceph is built on top of it but you can also use it as a standalone database.
(Disclaimer: ex-Ceph employee.)