Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I just want simple S3

blog.feld.me

Post reply on HN
Print view View original

31–40 of 130 posts

Re: I just want simple S3

#31

Post by CobrastanJorji »

tptacek wrote: post #27
CobrastanJorji wrote: post #21

This is an interesting write up, but I'm curious about the use case. If you don't need to scale, and you don't need to replicate anything, why do you want S3 specifically? Are you using a tool that wants to write to something S3-like? Do you just like reading and writing objects via HTTP POST and GET? Are you moving an app to or from the cloud?

It's probably the most important storage API in the industry. Implementing it gives you on-prem storage, AWS S3 (the Hoover Dam of Internet storage megaprojects, arguably the most reliable store of any kind available to any normal programmer), and a whole ecosystem of S3-compatible options with different features and price points. It's a little like asking why you'd use SQL.

The S3 standard is certainly really important. It's perhaps the most important web standard without any sort of standards organization or formal spec (seriously, Amazon, I'm begging you to open up to ISO or IEC or SNIA or somebody).

And SQL is also very important. And yet, if somebody said "I need to store data, but it's not relational, and I just need 1000 rows, what's the best SQL solution," I would still ask why exactly they needed SQL. The might have a good reason (for example, SQLite can be a weirdly good way to persist application data), but I don't know it yet. That's why I asked.

Re: I just want simple S3

#32

Post by colechristensen »

CobrastanJorji wrote: post #21

This is an interesting write up, but I'm curious about the use case. If you don't need to scale, and you don't need to replicate anything, why do you want S3 specifically? Are you using a tool that wants to write to something S3-like? Do you just like reading and writing objects via HTTP POST and GET? Are you moving an app to or from the cloud?

I want my application servers to be stateless and I've got state to keep that looks a lot more like files than database rows.

And I want things like backup, replication, scaling, etc. to be generic.

I wrote a git library implementation that uses S3 to store repositories for example.

Re: I just want simple S3

#33

Post by mickael-kerjean »

jerf wrote: post #15

I think we get a "S3 clone" about once every week or two on the Golang reddit. It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own". AI may have made this worse then ever.

Or maybe the underlying philosophy is different enough to warrant its own implementation. For example, the Filestash implementation (which I made) listed in the article is stateless and acts as a gateway that proxies everything to an underlying storage. We don't own the storage, you do, via any of the available connectors (SFTP, FTP, WebDAV, Azure, SMB, IPFS, Sharepoint, Dropbox, ...). You generate S3 keys bound to a specific backend and path, and everything gets proxied through. That's fundamentally different to not fit in the mold of other alternatives that mostly assume they own your storage and as a result can not be made stateless by design. That approach has pro and cons on each side

Re: I just want simple S3

#34

Post by evil-olive »

leosanchez wrote: post #3
ChromaticPanic wrote: post #2

Garage "unnecessarily complex" . If anything it's the simplest solution in the list especially compared to Ceph or Apache Ozone

Tried setting up rustfs today. It was easier that garagehq and it even comes with UI.

RustFS is the poster child in my mind for the worst kind of vibe-coded slop. it might be "simple" but it's not something I would ever trust with persistent data.

last year they had a security vulnerability where they allowed a hardcoded "rustfs rpc" token to bypass all authentication [0]

and even worse, if you read the resulting reddit thread [1] someone tracked down the culprit commits - it was introduced in July [2] and not even reviewed by another human before being merged.

then the fix 6 months later [3] mentions fixing a different security vulnerability, and seemingly only fixed the hardcoded token vulnerability by accident. that PR was also only reviewed by an LLM, not a human.

0: https://github.com/rustfs/rustfs/security/advisories/GHSA-h9...

1: https://www.reddit.com/r/selfhosted/comments/1q432iz/update_...

2: https://github.com/rustfs/rustfs/pull/163/

3: https://github.com/rustfs/rustfs/pull/1291

Re: I just want simple S3

#35

Post by jeroenhd »

jerf wrote: post #15

I think we get a "S3 clone" about once every week or two on the Golang reddit. It strikes me as a classic case of "we need all the interested people to pull in one project, not each start their own". AI may have made this worse then ever.

I'm pretty sure I set up most of what "Simple S3" using with Apache2 and WebDAV at least fifteen years ago.

Every month there's a post of "I just want a simple S3 server" and every single one of them has a different definition of "simple". The moment any project overlaps between the use cases of two "simple S3" projects, they're no longer "simple" enough.

That's probably why hosted S3-like services will exist even if writing "simple" S3 servers is so easy. Everyone has a different opinion of what basic S3 usage is like and only the large providers/startups with business licensing can afford to set up a system that supports all of them.

Re: I just want simple S3

#37

Post by ovaistariq »

larrymcp wrote: post #9
jockm wrote: post #5

Earlier quoted context omitted.

While not obvious from the article, it appears that they want something S3 like, but isn’t from Amazon, and possibly want to self host it. The article could be much more clear about the goals

Ah, thanks. Yeah I was confused because in his long list of vendors he didn't mention Wasabi, Backblaze etc. It appears that I do not know the context of his post.

or Tigris

Re: I just want simple S3

#38

Post by PunchyHamster »

S3 isn't "simple" tho.

It doesn't need to care about POSIX mess but there is whole swathes of features many implementations miss or are incomplete, both on frontend side (serving files with right headers, or with right authentication) and backend (user/policy management, legal hold, versioning etc.)

It gets even more messy when migrating, for example migrating your backups to garagefs will lose you versioning, which means that if your S3 secret used to write backups gets compromised, your backups are gone vs on implementation that supports versioning you can just rollback.

Similarly with password, some will give you secret and login but won't allow setting your own so you'd have to re-key every device using it, some will allow import, but only in certain format so you can restore from backup, bot not migrate from other software.

Re: I just want simple S3

#39

Post by PunchyHamster »

evil-olive wrote: post #34
leosanchez wrote: post #3

Earlier quoted context omitted.

Tried setting up rustfs today. It was easier that garagehq and it even comes with UI.

RustFS is the poster child in my mind for the worst kind of vibe-coded slop. it might be "simple" but it's not something I would ever trust with persistent data. last year they had a security vulnerability where they allowed a hardcoded "rustfs rpc" token to bypass all authentication [0] and even worse, if you read the resulting reddit thread [1] someone tracked down the culprit commits - it was introduced in July [2…

I recently submitted bug about how their own docs tell you to

* create rustfs user * run the rustfs from root via systemd, but with bunch of privileges removed * write logs into /var/logs/ instead of /var/log

Looks like someone told some LLM to make docs about running it as service and never looked at output

Re: I just want simple S3

#40

Post by nikeee »

evil-olive wrote: post #34
leosanchez wrote: post #3

Earlier quoted context omitted.

Tried setting up rustfs today. It was easier that garagehq and it even comes with UI.

RustFS is the poster child in my mind for the worst kind of vibe-coded slop. it might be "simple" but it's not something I would ever trust with persistent data. last year they had a security vulnerability where they allowed a hardcoded "rustfs rpc" token to bypass all authentication [0] and even worse, if you read the resulting reddit thread [1] someone tracked down the culprit commits - it was introduced in July [2…

I am building an S3 client [1] where I have a test matrix that tests against common S3 implementations, including RustFS.

That test matrix uncovered that post policies were only checked for exsitence and a valid signature, not if the request actually conforms to the signed policy. That was an arbitrary object write resulting in CVE-2026-27607 [2].

In the very first issue for this bug [3], it seemed that the authors of the S3 implementation didn't know the difference between the content-length of GetObject and content-length-range of a PostObject. That was kind of a bummer and leads me to advise all my friends not to use rustfs, though I like what they are doing in principal (building a Minio alternative).

[1]: https://github.com/nikeee/lean-s3 [2]: https://github.com/rustfs/rustfs/security/advisories/GHSA-w5... [3]: https://github.com/rustfs/rustfs/issues/984

Post reply on HN