Live data from Hacker News

In S3 simplicity is table stakes

allthingsdistributed.com

51–60 of 142 posts

Re: In S3 simplicity is table stakes

#51
post #44

S3 was one of the first offerings coming out of AWS right? It’s pretty legendary and a great concept to begin with. You can tell by how much sense it makes and then trying to wrap your ahead around the web dev world pre-S3.

? The web dev world pre-S3 was pretty much the same, but you stored your files on a regular server (and set up your own redundancy and backup strategy). Not that much different to be honest from an end user's point of view.

Re: In S3 simplicity is table stakes

#52
post #10

I have a feeling that economies of scale have a point of diminishing returns. At what point does it become more costly and complicated to store your data on S3 versus just maintaining a server with RAID disks somewhere? S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.

There are a few stories from companies that moved away from S3, like Dropbox, and who shared their investments and expenses.

The long and short of it is that getting anywhere near the redundancy, reliability, performance etc of S3, you're spending A Lot of money.

Re: In S3 simplicity is table stakes

#53
post #46
post #34

I found out last year that you can actually run a full SPA using S3 and a CDN. It’s kind of a nuts platform

I use S3+Cloudfront for static sites and Cloudflare workers if it needed. It's always crazy to me that people will run a could be static site on Netlify/Vercel/etc.

We've used Netlify at previous projects, we used it because it was easy. No AWS accounts or knowledge needed, just push to master, let the CI build (it was a Gatsby site) and it was live.

Re: In S3 simplicity is table stakes

#54
Lots of comments here talking about how great S3 is.

Anyone willing to give a cliff notes about what's good about it?

I've been running various sites and apps for a decade, but have never touched S3 because the bandwidth costs are 1, sometimes 2 orders of magnitude more expensive than other static hosting solutions.

Re: In S3 simplicity is table stakes

#55
post #10

I have a feeling that economies of scale have a point of diminishing returns. At what point does it become more costly and complicated to store your data on S3 versus just maintaining a server with RAID disks somewhere? S3 is an engineering marvel, but it's an insanely complicated backend architecture just to store some files.

There is a diminishing return of what percentage you save, sure. But amazon will always be at that edge. They already have amortized the equipment, labour, administration, electricity, storage, cooling, etc.

They also already have support for storage tiering, replication, encryption, ACLs, integration with other services (from web access to sending notifications of storage events to lambda, sqs, etc). Uou get all of this whether you're saving 1 eight bit file or trillions of gigabyte sized ones.

There are reasons why you may need to roll your own storage setup (regulatory, geographic, some other unique reason), but you'll never be more economical than S3, especially if the storage is mostly sitting idle.

Re: In S3 simplicity is table stakes

#56
post #46

Earlier quoted context omitted.

I use S3+Cloudfront for static sites and Cloudflare workers if it needed. It's always crazy to me that people will run a could be static site on Netlify/Vercel/etc.

We've used Netlify at previous projects, we used it because it was easy. No AWS accounts or knowledge needed, just push to master, let the CI build (it was a Gatsby site) and it was live.

I think Netlify is great but to me it's overkill if you just have a static site.

I understand that Netlify is much simpler to get started with and setting up an AWS account is somewhat more complex. If you have several sites, it's worth spending the time to learn.

Re: In S3 simplicity is table stakes

#57

Lots of comments here talking about how great S3 is. Anyone willing to give a cliff notes about what's good about it? I've been running various sites and apps for a decade, but have never touched S3 because the bandwidth costs are 1, sometimes 2 orders of magnitude more expensive than other static hosting solutions.

S3 is great for being able to stick files somewhere and not have to think about any of the surrounding infrastructure on an ongoing basis [1]. You don't have to worry about keeping a RAID server, swapping out disks when one fails, etc.

For static hosting, it's fine, but as you say, it's not necessarily the cheapest, though you can bring the cost down by sticking a CDN (Cloudflare/CloudFront) in front of it. There are other use cases where it really shines though.

[1]: I say ongoing basis because you will need to figure out your security controls, etc. at the beginning so it's not totally no-thought.

Re: In S3 simplicity is table stakes

#60
post #38
post #19

Earlier quoted context omitted.

It’s designed for that level of durability, but it’s only as good as a single change or correlated set of hardware failures that can quickly change the theoretical durability model. Or even corrupting data is possible too.

You're totally correct, but these products also need to be specifically designed against these failure cases (i.e. it's more than just MTTR + MTTF == durability). You (of course) can't just run deployments without validating that the durability property is satisfied throughout the change.

Yep! There’s a lot of checksum verification, carefully orchestrated deployments, hardware diversity, erasure code selection, the list goes on and on. I help run a multi-exabyte storage system - I’ve seen a few things.
Post reply on HN