11–20 of 82 posts
I don't get it, if it's running on the same (mentioning "local") machine, why does it even need the S3 API? Could just be plain IO on the local drive(s)
The S3 API doesn't work like normal filesystem APIs. Part of it is that it follows the object storage model, and part of it is just to lock people into AWS once they start working with it.
This is some next-level conspiracy theory stuff. What exactly would the alternative have been in 2006? S3 is one of the most commonly implemented object storage APIs around, so if the goal is lock-in, they're really bad at it.
I don't get it, if it's running on the same (mentioning "local") machine, why does it even need the S3 API? Could just be plain IO on the local drive(s)
The S3 API doesn't work like normal filesystem APIs. Part of it is that it follows the object storage model, and part of it is just to lock people into AWS once they start working with it.
I'm sure it's a lot better now but everytime I see btrfs I get PTSD.
But a quick grep across versitygw tells me they don't use Sync()/fsync, so not a problem... Any data loss occurring from that is obviously not btrfs fault.
I don't get it, if it's running on the same (mentioning "local") machine, why does it even need the S3 API? Could just be plain IO on the local drive(s)
So you don't need to refactor your code?
For this project, where you have 120GB of customer data, and thirty requests a second for ~8k objects (0.25MB/s object reads), you’d seem to be able to 100x the throughput vertically scaling on one machine with a file system and an SSD and never thinking about object storage. Would love to see why the complexity
Earlier quoted context omitted.
The S3 API doesn't work like normal filesystem APIs. Part of it is that it follows the object storage model, and part of it is just to lock people into AWS once they start working with it.
I'm 100% aware of how S3 works. I was questioning why the S3 API is needed when the service is using local storage.
I've worked at a few places where single-node K8s "clusters" were frequently used just because they wanted the same API everywhere.
Self Hosted object storage looks neat! For this project, where you have 120GB of customer data, and thirty requests a second for ~8k objects (0.25MB/s object reads), you’d seem to be able to 100x the throughput vertically scaling on one machine with a file system and an SSD and never thinking about object storage. Would love to see why the complexity
Earlier quoted context omitted.
The S3 API doesn't work like normal filesystem APIs. Part of it is that it follows the object storage model, and part of it is just to lock people into AWS once they start working with it.
> part of it is just to lock people into AWS once they start working with it. This is some next-level conspiracy theory stuff. What exactly would the alternative have been in 2006? S3 is one of the most commonly implemented object storage APIs around, so if the goal is lock-in, they're really bad at it.
I don't get it, if it's running on the same (mentioning "local") machine, why does it even need the S3 API? Could just be plain IO on the local drive(s)
The S3 API doesn't work like normal filesystem APIs. Part of it is that it follows the object storage model, and part of it is just to lock people into AWS once they start working with it.