This is an interesting write up, but I'm curious about the use case. If you don't need to scale, and you don't need to replicate anything, why do you want S3 specifically? Are you using a tool that wants to write to something S3-like? Do you just like reading and writing objects via HTTP POST and GET? Are you moving an app to or from the cloud?
I just want simple S3
21–30 of 130 posts
Re: I just want simple S3
#22I think we get a "S3 clone" about once every week or two on the Golang reddit. It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own". AI may have made this worse then ever.
Diverse competition is the best way to identify a winning formula, which can then be perfected by a fewer number of players.
Re: I just want simple S3
#23I made https://github.com/uroni/hs5 -- focus is on single node and high performance. So plenty of alternatives available.
Re: I just want simple S3
#24microceph is pretty nice and straightforward for throwaway s3 endpoints https://canonical-microceph.readthedocs-hosted.com/stable/tu...
Has anyone that has set up microceph determined the overhead of the required multiple OSDs? The docs make it sound scary, but it's not clear if that's because people run it on a Pi with an sdcard for block storage or because someone once ran 18TB of OSDs in production that then fell over.
I do continue to be impressed/ over-awed by how effectively scared the Ceph docs are about just how many system resources you need. To run a mid tier not that fast storage cluster. Bother.
Impressive as hell software and I am so glad to have it. But man! The insistence on mountains of ram per TB, on massive IO is intimidating.
Re: I just want simple S3
#25S3-compatible storage solution: https://www.hetzner.com/storage/object-storage/
Re: I just want simple S3
#26I think we get a "S3 clone" about once every week or two on the Golang reddit. It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own". AI may have made this worse then ever.
I think it's like NES emulators. It's not that anyone needs one more. It's just that they're fun to make.
Re: I just want simple S3
#27This is an interesting write up, but I'm curious about the use case. If you don't need to scale, and you don't need to replicate anything, why do you want S3 specifically? Are you using a tool that wants to write to something S3-like? Do you just like reading and writing objects via HTTP POST and GET? Are you moving an app to or from the cloud?
It's probably the most important storage API in the industry. Implementing it gives you on-prem storage, AWS S3 (the Hoover Dam of Internet storage megaprojects, arguably the most reliable store of any kind available to any normal programmer), and a whole ecosystem of S3-compatible options with different features and price points.
It's a little like asking why you'd use SQL.
Re: I just want simple S3
#28S3-compatible storage solution: https://www.hetzner.com/storage/object-storage/
They want to run it locally.
Re: I just want simple S3
#29Why do rust compile times matter for a production deployment?
Re: I just want simple S3
#30Since this has come up 4-5 times on the thread already, the clear subtext of this post is that this developer wants to build to the S3 API, but run their storage locally --- maybe for testing reasons, maybe for data hygiene reasons, maybe for performance reasons. So things like "what about Hugging Face's object storage product" don't really answer their question.