Live data from Hacker News

Please fix the AWS free tier before somebody gets hurt

cloudirregular.substack.com

271–280 of 457 posts

Re: Please fix the AWS free tier before somebody gets hurt

#271

Earlier quoted context omitted.

AWS employs a whole lot of brilliant people, and they're an insanely profitable business unit. I'm sure those people, given the appropriate motivation and opportunity, could provide a solution (or several) that avoids foot-gun data loss at every turn. There's all sort of solutions they could go with. As an idea, perhaps some kind of functionality within the billing side of things: They could, perhaps, decide to just…

It's a good lesson in cloud computing to know how much you're spending on services. Usually if you're a single person and you leave an expensive product running unknowingly, AWS support have been kind and refund you the money and take the loss as 'customer made an error' type transaction. It's not like they hold the debt over your head for life. I think the article is a bit extreme to highlight the person's reaction,…

The problem is only partially "know how much you're spending on services".

There's a big problem of: There's a huge amount of AWS that even if you configure it "right" - still leaves you with potentially unlimited liability.

Take a simple example of using Cloudfront and S3 together to host a basic website. There's a good well documented way of securing this all "right".

However that doesn't protect you from "slashdotting" - perhaps you put an image on your site and it gets linked from the frontpage of Reddit.

You go from paying a few $/month, to suddenly overnight having potentially hundreds or thousands of dollars of spend because tens of millions of people have viewed your image.

AWS Billing alerts are not going to save you, because once they (eventually, maybe a day or so late) fire, and assuming you immediately see that notification, figure out what it is being accessed, and remove that file (or just turn off the bucket/cloudfront entirely) - the bills could have gone way up.

It gets worse when you're talking about other AWS services.

The whole point being that billing caps, along with billing alerts that are actually timely, are actually necessary.

You shouldn't have to depend on the grace of AWS Customer Service to not force you to decide whether to pay rent, eat, or your massive credit card bill.

> Realistically though, if you can't afford $20, how on earth did you afford a credit card to sign up for AWS in the first place?

The example was $200. And for a lot of people, particularly students and other people new to development - $200 can be a large chunk of money.

Part of the problem is that you go to AWS Conferences and talks, and they talk about how there's all this amazing stuff, and encourage people to sign up and try it out.

What they never do is tell you "Oh, by the way, this demo costs $x/hour". I'd love to see some kind of taxi-meter style box behind the presenter/in the corner of webcasts showing how much the AWS Bill would be for the demo.

Re: Please fix the AWS free tier before somebody gets hurt

#272

Earlier quoted context omitted.

Is it reasonable to expect a brand new aws user to know how to do those things?

The same as it is not expected by anyone to know how to ride a bike or drive a car. Someone has to teach you or you have to go and take a course. Same with any technology, you should read first all the documentation that there is, watch all the videos that AWS releases every year for free where they explain in detail every service that they have, and try to explain the best practices of it - this is all for free, or…

[deleted]

Re: Please fix the AWS free tier before somebody gets hurt

#273

Earlier quoted context omitted.

Is it reasonable to expect a brand new aws user to know how to do those things?

The same as it is not expected by anyone to know how to ride a bike or drive a car. Someone has to teach you or you have to go and take a course. Same with any technology, you should read first all the documentation that there is, watch all the videos that AWS releases every year for free where they explain in detail every service that they have, and try to explain the best practices of it - this is all for free, or…

You can by insure for you as a driver and for your car (and in many jurisdictions have to) — this limits your responsibility. You can spend $10000 on AWS and still make a mistake and pay another $10k+.

I don’t get the point that AWS docs are free. I mean, they are free, but the services are not. Do you pay for a visit in a car dealership?

Re: Please fix the AWS free tier before somebody gets hurt

#274
post #30

Earlier quoted context omitted.

I’ve heard of plenty of people accidentally blasting past the free tier, or past dev credits, or other ways where they got burned by a surprise bill. 100% of those people have gotten a refund from AWS support by filing a ticket. AWS pretty clearly isn’t raking in money on this. Even at your imaginary 2 million a year, that’s a drop in the bucket for AWS, let alone Amazon.

It's got to be Hanlon's Razor here, I doubt AWS is twirling their mustache and cackling maniacally. It's probably a difficult problem to solve and not easily attachable to a profit center or something, so nobody feels empowered to actually fix it. I'm sure some crusader within AWS could get it done if they tried hard enough and collected the data to show the negative side effects and how they effect AWS. An article l…

One person isn’t going to come remotely close to fixing this. I work for AWS/Azure/GCP and can tell you first hand that billing is an insanely hard problem to solve. It’s all done in batch by the service itself and pushed to a centralized service once all the hard work has been done. Each service has a small, underfunded team that handles billing and their last thought is some random guy spending a few grand on his personal account. Every ounce of their energy is going to reducing their own COGs and reacting to the asks of the biggest customers.

Re: Please fix the AWS free tier before somebody gets hurt

#275
post #170

Earlier quoted context omitted.

“Small, cheap things like "a single DynamoDB query" or "a byte of bandwidth" are often also high-volume, and you don't want to introduce billing checks to critical paths for reliability / latency / cost reasons” That’s hardly necessary. Let’s suppose you have some service that costs 1 cent every 1,000 queries. If you’re billing it then you need to be keeping track of it and incrementing some counter somewhere. If old…

That counter may well not exist outside of billing for longer than it takes to batch some records together. It will need to be shared and synchronized with the rest of the fleet, the other fleets in the availability zone, the other zones in the region, the other regions, and every other service in AWS. There are strict rules about crossing these boundaries for sake of failure isolation. As an amusing extra sticking p…

>your phone is only attached to one tower at a time

Not when you're being simswapped

Re: Please fix the AWS free tier before somebody gets hurt

#276

Earlier quoted context omitted.

The same as it is not expected by anyone to know how to ride a bike or drive a car. Someone has to teach you or you have to go and take a course. Same with any technology, you should read first all the documentation that there is, watch all the videos that AWS releases every year for free where they explain in detail every service that they have, and try to explain the best practices of it - this is all for free, or…

You can by insure for you as a driver and for your car (and in many jurisdictions have to) — this limits your responsibility. You can spend $10000 on AWS and still make a mistake and pay another $10k+. I don’t get the point that AWS docs are free. I mean, they are free, but the services are not. Do you pay for a visit in a car dealership?

> You can by insure for you as a driver and for your car (and in many jurisdictions have to) — this limits your responsibility

Real insurance policies have maximum coverage limits, so, while it reduces your liability by a capped amount, it doesn’t, strictly speaking, limit it; you still face unbounded potential liability beyond your insurance coverage.

Re: Please fix the AWS free tier before somebody gets hurt

#277
post #208

Earlier quoted context omitted.

I also work at AWS (nowhere near billing), so the usual disclaimers apply, but: I actually have no idea if billing is real-time or not? I think it's mostly batch, but the records aren't necessarily, though they may be aggregated a bit. The general point in this thread certainly holds: our systems provide service first, bill second, and that by throwing a record over the wall to some other internal system. It's not un…

There’s a lot of middle ground between credit limit checks within every database transaction and the current state.

> There’s a lot of middle ground between credit limit checks within every database transaction and the current state.

But there isn’t a lot of middle ground between distributed, strongly-consistent credit limit checks every API call and billing increment (which is, IIRC, instance-seconds or smaller for some time-based services) and a hard billing cap that is actually usable on a system structured like AWS. Partial solutions reduce the risk but don’t eliminate the problem, and at AWS scale reducing the risk means you still have significant numbers of people reliant on the “call customer service” mitigation, and how much spending and system compromise to narrow this billing issue is worthwhile if you are still in that position?

Re: Please fix the AWS free tier before somebody gets hurt

#278
post #56

Earlier quoted context omitted.

A spending kill switch can be setup on AWS using AWS Budgets alerts and Lambda but it’s a DIY project, not a built-in feature.

If it can be done DIY, it can be done first party.

If its done first party, someone (realistically, af AWS’ scale, a substantial number of someones) will mess up using it and nuke their account.

Customer service can correct “we screwed up and have a giant bill” easier than “we screwed up and lost all our data”.

So its not going to happen first party.

(It’s also not really possible DIY as a hard certain cutoff at or before a limit, only at an indefinite interval in time and money beyond a limit. So you still have potentially unbounded expense.)

Re: Please fix the AWS free tier before somebody gets hurt

#279
Yes, there should be easier ways for learners to experiment with AWS without risk. In general, people should be made aware of how much something could cost them before they can agree with it. We have regulations towards this end when it comes to investments. Huge unexpected bills used to be a big problem with phone companies, and it took a lot of effort to move past it. I hope the cloud industry can make progress in this direction.

What I'm going to say next should not be taken to detract from that in any way.

(And I know you don't want to hear it, and I could just save everyone some trouble and not say it. But I'm going to say it.)

When I read this:

> please help I made a ticket and called support but i really need to make sure this is dead please Im 20 i really dont have $200 for them please help

I honestly did a double take when I got to the "20". If you'd put a blank in there and asked me to fill it, I would have said "13". I'm still not 100% sure that the blogger didn't alter the age to obfuscate personal details.

I don't mean to single out this person as especially immature, quite the opposite: the interesting thing is that they assume that a 20 year old is obviously a smol bean who could not possibly be expected to figure out AWS billing or come up with $200, _and we agree_. The notion didn't stand out as surprising to the blogger, or to any of the other commenters on HN so far.

I'm not saying they're wrong! I just find this a remarkable signal of how far we, as a culture, have gone in extending childhood well into college age.

Re: Please fix the AWS free tier before somebody gets hurt

#280

AWS employs cost obfuscation by design otherwise the default view when you open the console would show you all of your current active services. Not only is that not the case, a single screen to show you all of your current active services doesn't exist. You need to take a deep dive into cost explorer (assuming you have access in corporate land) and try to decipher in what that all means.

AWS is literally trying to trick you. With friend like these, who needs enemies?

The Trough of Disillusionment for AWS is going to be Bastille Day levels of ugly.

Post reply on HN