21–30 of 147 posts
What is the difference between a "distributed object storage" and a file system?
Object store = store blobs of bytes. Usually by bucket + key accessible over HTTP. No POSIX expectation.
Distributed = works spread across multiple servers in different locations.
I don’t understand why everyone wants to replicate AWS APIs for things that are not AWS. S3 is a horrible interface with a terrible lack of features. It’s just file storage without any of the benefits of a file syste - no metadata, no directory structures, no ability to search, sort, or filter. Combine that with high latency network file access and an overly verbose API. You literally have a bucket for storing files,…
Does your remote file server magically avoid network latency? Mine doesn’t.
In case you didn’t know, inside the bucket you can use a full path for S3 files. So you can have directories or folders or whatever.
Some benefits of this system (KV style access) is to support concurrent usage better. Not every system needs it, but if you’re using an object store you might.
SeaweedFS is great as well. https://github.com/seaweedfs/seaweedfs
Earlier quoted context omitted.
Because at this point it's a well known API. I bet people want to recreate AWS without the Amazon part, and so this is for them. Which, to your point, makes no sense because as you rightly point out, people use S3 because of the Amazon services and ecosystem it is integrated with - not at all because it is "good tech"
S3 was the second AWS service, behind SQS, and saw rapid adoption which cannot be explained by integration with services introduced later.
I have used Garage for a long time. It's great, but the AWS sigv4 protocol for accessing it is just frustrating. Why can't I just send my API key as a header? I don't need the full AWS SDK to get and put files, and the AWS sigv4 is a ton of extra complexity to add to my projects. I don't care about the "security benefits" of AWS sigv4. I hope the authors consider a different authentication scheme so I can recommend G…
I don’t understand why everyone wants to replicate AWS APIs for things that are not AWS. S3 is a horrible interface with a terrible lack of features. It’s just file storage without any of the benefits of a file syste - no metadata, no directory structures, no ability to search, sort, or filter. Combine that with high latency network file access and an overly verbose API. You literally have a bucket for storing files,…
Implementing a filesystem versus an object store involves severe tradeoffs in scalability and complexity that are rarely worth it for users that just want a giant bucket to dump things in.
The API doesn't matter that much, but everything already supports S3, so why not save time on client libraries and implement it? It's not like some alternative PUT/GET/DELETE API will be much simpler-- though naturally LIST could be implemented myriad ways.
SeaweedFS is great as well. https://github.com/seaweedfs/seaweedfs
Edit: memory = RAM