Live data from Hacker News

AWS S3: Sometimes you should press the $100k button

cyclic.sh

51–60 of 240 posts

Re: AWS S3: Sometimes you should press the $100k button

#51

Can someone explain what happened in the end? From my understanding nothing happened (they deprioritizod the story for fixing it) and they are still blowing through the cloud budget.

How I read the article, nothing happened. I think it is a cautionary tale of why you should probably bite the bullet and press the button instead of doing the "easier" thing which ends up being harder and more expensive in the end.

Re: AWS S3: Sometimes you should press the $100k button

#52
post #22

Your website renders as a big empty blue page in Firefox unless I disable tracking protection (and in my case, since I have noscript, I have to enable javascript for "website-files.com", a domain that sounds totally legit).

The problem is that the DIV that contains the main text has the attribute 'style="opacity:0"'. Presumably, this is something that the JavaScript turns off.

A lot of sites like to do things like this for some reason. I haven't figured out why. I like to use Stylus to mitigate these if I can, rather than enabling JavaScript.

Re: AWS S3: Sometimes you should press the $100k button

#53

I have caused billing spikes like this before those little warnings were invented and it was always a dark day. They are really a life saver. Lifecycle rules are also welcome. Writing them yourself was always a pain and tended to be expensive with list operations eating up that api calls bill. ---- Once I supported an app that dumped small objects into s3 and begged the dev team to store the small objects in oracle a…

> I then went for a compromise solution asking if we could stitch the small objects together after a period of time so they would be eligible for things like infrequent access or glacier but, alas, "dev time is expensive you know" so N figure s3 bills continue as far as I know.

This hits home so hard that it hurts. In my case is not S3 but compute bills but the core concept is the same.

Re: AWS S3: Sometimes you should press the $100k button

#54
post #12

TL-DR: Object stores are not databases. Don't treat them like one.

Try telling that to developers; they love using S3 as both a database and a filesystem. It's gotten to the point where we need a training for new devs to tell them what not to do in the cloud.

do you know if such sources exist publicly? I would be most interested in perusing recommended material on the subject.

Re: AWS S3: Sometimes you should press the $100k button

#55
post #45

sigh . My team is facing all these issues. Drowning in data. Crazy S3 bill spikes. And not just S3 - Azure, GCP, Alibaba, etc since we are a multi-cloud product. Earlier, we couldn't even figure out lifecycle policies to expire objects since naturally every PM had a different opinion on the data lifecycle. So it was old-fashioned cleanup jobs that were scheduled and triggered when a byzantine set of conditions were m…

The data protection regulations really are so freeing, huh. It's amazing to be able to delete all this stuff without worrying about having to keep it forever.

Re: AWS S3: Sometimes you should press the $100k button

#57
post #5

Off topic: for people with a "million billion" objects, does the S3 console just completely freeze up for you? I have some large buckets that I'm unable to even interact with via the GUI. I've always wondered if my account is in some weird state or if performance is that bad for everyone. (This is a bucket with maybe 500 million objects, under a hundred terabytes)

Sort of related, I faced such an issue when I had a gui table that was triggering a count on a large object set via sql so it could display the little "1 to 50 of 1000000". This is presumably why services like google say "of many". Wonder if they have a similar issue.

Re: AWS S3: Sometimes you should press the $100k button

#58
post #52
post #22

Your website renders as a big empty blue page in Firefox unless I disable tracking protection (and in my case, since I have noscript, I have to enable javascript for "website-files.com", a domain that sounds totally legit).

The problem is that the DIV that contains the main text has the attribute 'style="opacity:0"'. Presumably, this is something that the JavaScript turns off. A lot of sites like to do things like this for some reason. I haven't figured out why. I like to use Stylus to mitigate these if I can, rather than enabling JavaScript.

[deleted]

Re: AWS S3: Sometimes you should press the $100k button

#59
On this topic, it's always surprising to me how few people even seem to know about different storage classes on S3...or even intelligent tiering (which I know carries a cost to it, but allows AWS to manage some of this on your behalf which can be helpful for certain use-cases and teams).

We did an analysis of S3 storage levels by profiling 25,000 random S3 buckets a while back for a comparison of Amazon S3 and R2* and nearly 70% of storage in S3 was StandardStorage which just seems crazy high to me.

* https://www.vantage.sh/blog/the-opportunity-for-cloudflare-r...

Re: AWS S3: Sometimes you should press the $100k button

#60
The AWS horror stories never cease to amaze me. It's like we're banging our heads against the wall expecting a different outcome each time. What's more frustrating, the AWS zealots are quite happy to tell you how you're doing it wrong. It's the users fault for misusing the service. The reality is, AWS was built for a specific purpose and demographic of user. It's now complexity and scale makes it unusable for newer devs. I'd argue, we need a completely new experience for the next generation.
Post reply on HN