Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I just want simple S3

blog.feld.me

Post reply on HN
Print view View original

121–130 of 130 posts

Re: I just want simple S3

#121

Post by iExalt »

patates wrote: post #83
PunchyHamster wrote: post #38

S3 isn't "simple" tho. It doesn't need to care about POSIX mess but there is whole swathes of features many implementations miss or are incomplete, both on frontend side (serving files with right headers, or with right authentication) and backend (user/policy management, legal hold, versioning etc.) It gets even more messy when migrating, for example migrating your backups to garagefs will lose you versioning, which…

S3 needs a split: QS3 (Quite Simple Storage Service) for the barebones. Bucket/Object CRUD. Maybe: Multipart Uploads. Presigned URLs. S3 for Object Tagging, Access Control Lists, etc. S3E (enterprise? extended? elaborate?) for Object Lock & Retention (WORM compliance, Legal Holds), Event Notifications and so on.

Proposal to rename the services:

* S4: Stupid/Silly Simple Storage Service

* S3: Simple Storage Service

* S2: Storage Service

Re: I just want simple S3

#122

Post by nitwit005 »

tptacek wrote: post #30

Since this has come up 4-5 times on the thread already, the clear subtext of this post is that this developer wants to build to the S3 API, but run their storage locally --- maybe for testing reasons, maybe for data hygiene reasons, maybe for performance reasons. So things like "what about Hugging Face's object storage product" don't really answer their question.

It feels like there is more missing context than that:

> I just want S3. My needs are pretty basic. I don't need to scale out. I don't need replication. I just need something that can do S3 and is reliable and not slow.

What does reliable mean, without replication, and no mention of backups?

Re: I just want simple S3

#123

Post by bluefirebrand »

iExalt wrote: post #121
patates wrote: post #83

Earlier quoted context omitted.

S3 needs a split: QS3 (Quite Simple Storage Service) for the barebones. Bucket/Object CRUD. Maybe: Multipart Uploads. Presigned URLs. S3 for Object Tagging, Access Control Lists, etc. S3E (enterprise? extended? elaborate?) for Object Lock & Retention (WORM compliance, Legal Holds), Event Notifications and so on.

Proposal to rename the services: * S4: Stupid/Silly Simple Storage Service * S3: Simple Storage Service * S2: Storage Service

You can't have the numbers go down like this. Try pitching S2 to your project manager. You will be told 100% "Why not use S4 instead, that sounds better"

Re: I just want simple S3

#124

Post by rglover »

sonnyz wrote: post #12
panarky wrote: post #8

Sounds like you want S4. Super simple storage service.

Listen to this: 7... Minute... Abs. You walk into a video store and you see 8 minute abs and 7 minutes abs. Which one are you gonna buy?

"Step into my office?"

"Why?"

"Because you're fucking fired!"

Re: I just want simple S3

#126

Post by pythonaut_16 »

syabro wrote: post #87
thayne wrote: post #73

Earlier quoted context omitted.

I don't think that is what they are looking for. They just want something with an s3 compatible API they can run on their local network or maybe even on the same host.

what's the point then? Just api around FS?

I use a local garagefs on my NAS for small/new side projects, and it’s on my Tailscale for easy access

- Lets me deploy stateless containers easily

- Let’s me leverage the NAS for local redundancy and a more centralized place to do backups

- When a project grows it’s easy to promote it to use a hosted S3

- Local S3 becomes a target for Litestream and Restic

- Developing against the local fs and then handling file storage is a huge friction, unless I’m using something like Rails that already has a good abstraction

Re: I just want simple S3

#129

Post by bonesss »

thayne wrote: post #118
bonesss wrote: post #116

Earlier quoted context omitted.

So, why not write to a shared wrapper/facade? If you split the interaction API out to an interface detailing actual program interaction with the service, then write an s3 backend and an FS backend. Then you could plug in any backend as desired and write agnostic application code. Personally I end up there anyways testing and specifying the third party failure modes.

What if you need it because you are using a third party application that requires an s3 api? Or you want to test your code that interacts with an s3 API?

Same answer, wrap you usage and use the wrapper for agonistic testing & coding.

That’s the entire point: my domain code ‘stores’ something, my implementation decides if that is S3 or a fixed kind of error or writes to a disc.

If my domain uses a third party app that needs an S3 API, and we see not pointing it at S3, then I am writing an s3 API to test that dependency and ensure the third party code works regardless. Normally I’d call that “their problem” and call their service with the same wrapper as above.

Post reply on HN