Please fix the AWS free tier before somebody gets hurt • Hacker News Board

Re: Please fix the AWS free tier before somebody gets hurt

#251

Post by czgs2woh4ue3 » Wed, May 05, 2021, 5:13 AM UTC

KirillPanov wrote: post #241

NathanKP wrote: post #106

Earlier quoted context omitted.

This is my personal opinion, though I do work at AWS: It's not that real time counting is difficult, it is that the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical. There is a reason why banks and financial institutions generally do batch processing in the off peak hours when electricity is cheaper and there is less demand for the compute…

> the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical You don't have to bill in real time. You just have to provision funding for every resource except network bandwidth. Customer sets a monthly spend limit. Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly…

> Update that setting in your routers (qdisc in Linux) as part of the API call that allocated a resource. If you claim your routers don't have a limit like this I call shenanighans.

Eh. AWS's edge network is highly distributed. Unless you want an even split of your rate limit across every possible way out of the network, you'd be much better off settling for an even split across your EC2 instances, and there's no room for bursting in this model. Enforcing per-instance limits (on any dimension) sounds pretty feasible, though.

This wouldn't generalize straightforwardly to services that don't have obvious choke points that can impose this sort of throttling, such as, I think, DynamoDB.

Re: Please fix the AWS free tier before somebody gets hurt

#252

Post by theshrike79 » Wed, May 05, 2021, 5:15 AM UTC

singlow wrote: post #235

artichokeheart wrote: post #24

AWS employs cost obfuscation by design otherwise the default view when you open the console would show you all of your current active services. Not only is that not the case, a single screen to show you all of your current active services doesn't exist. You need to take a deep dive into cost explorer (assuming you have access in corporate land) and try to decipher in what that all means.

I don't understand why so many people want to use AWS for such small budgets. At that scale wouldn't it be easier to just build everything out of a few VMs at a cheap service? AWS is awesome when you have a large number of resources, that are created programmatically and reproducibly, with redundancy and duplicate environments. The budgeting tools are really amazing at letting you categorize your costs and create app…

> At that scale wouldn't it be easier to just build everything out of a few VMs at a cheap service?

I have a handful of AWS Lambda functions with a DynamoDB backend serving hundreds of clients, my bill for the month of April was $0.01.

No, a VM wouldn't cut it.

But you are right that there are certain slots where AWS doesn't make sense: There's one in the lower middle range where you can save a bunch of money by using a VM or two with your own DB servers. And there's the one where you're so big it might actually be worth it to implement the whole stack yourself.

Re: Please fix the AWS free tier before somebody gets hurt

#253

Post by whoknowswhat11 » Wed, May 05, 2021, 5:21 AM UTC

jiggawatts wrote: post #136

Everyone: PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits. As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits. There's a difference between "unable" and "unwilling". The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't s…

What a lie

AWS offers an AWS educate account already - go here to learn more

https://aws.amazon.com/education/awseducate/aws-educate-faqs...

If you are a startup go through AWS activate - 100,000 to play with.

GCP etc al do not offer this either. This is not done crime- AWS pioneered this free tier model. I used Dell and you paid for everything there.

Re: Please fix the AWS free tier before somebody gets hurt

#254

Post by dragonwriter » Wed, May 05, 2021, 5:29 AM UTC

KirillPanov wrote: post #241

NathanKP wrote: post #106

Earlier quoted context omitted.

This is my personal opinion, though I do work at AWS: It's not that real time counting is difficult, it is that the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical. There is a reason why banks and financial institutions generally do batch processing in the off peak hours when electricity is cheaper and there is less demand for the compute…

> the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical You don't have to bill in real time. You just have to provision funding for every resource except network bandwidth. Customer sets a monthly spend limit. Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly…

> Customer sets a monthly spend limit. Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly cost of that new thing from their spend limit. If the spend limit would go negative, you refuse to create the new resource.

AWS systems are highly distributed; this kind of sharp billing cap would necessarily introduce a new strong consistency requirement across multiple services, many of which aren’t even strongly consistent considered one at a time (and that’s often true even if you limit to a single region.)

> Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly cost of that new thing from their spend limit

For the motivating use case (avoiding a bill on the scale of even $200—possibly even $1—from a free-tier-eligible account), using monthly chunks doesn’t work; you suddenly couldn’t spin up a second simultaneous EC2 instance of ant kind after an initial t3.micro instance, which would cutoff many normal free tier usage patterns.

I mean, that’s a good way of capping if you are using AWS as a super overpriced steady-state VPS, but that’s not really the usage pattern that causes the risks that the cap idea is intended to protect against.

This is a particularly poor solution to completely the wrong problem.

Re: Please fix the AWS free tier before somebody gets hurt

#255

Post by blabitty » Wed, May 05, 2021, 5:34 AM UTC

davidgatti wrote: post #249

jiggawatts wrote: post #136

Everyone: PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits. As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits. There's a difference between "unable" and "unwilling". The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't s…

Hmm not sure where the problem is. With IAM you can create a policy to limit the user to only use certain services, deploy only the cheapest type of instance type, limit the region, the time of day etc. All of this and more is built in to IAM by default. You can even create a CloudFormation as a internal product this way you can limit even more what people do, and budget for that. And on top of that you can make a La…

Is it reasonable to expect a brand new aws user to know how to do those things?

Re: Please fix the AWS free tier before somebody gets hurt

#256

Post by solatic » Wed, May 05, 2021, 5:35 AM UTC

sl1ck731 wrote: post #178

grioghar wrote: post #151

Earlier quoted context omitted.

How are people not building in Terraform for an easy ‘destroy’ at the end? I know it’s rhetorical and a lesson learned myself, but yeah… I would expect folks to learn to use this tool to help manage costs this way.

Typically Terraform takes longer to get something working than mindlessly clicking through the console. In my experience those mindless clickthrough things end up sticking around for years even when they weren't intended to.

This is why you have separate development and production accounts: a development account where you mindlessly click through so that you can learn through the UI what's available and how it works; cleaned up on a regular basis by something like aws-nuke, and a production account where you have the discipline to only create resources through Terraform / CloudFormation etc.

Re: Please fix the AWS free tier before somebody gets hurt

#257

Post by uglygoblin » Wed, May 05, 2021, 5:40 AM UTC

AWS was charging me 0.50¢ a month for over a year for an account that was tied to a Google Suite email address for a failed startup I was part of.

I couldn't recover access to the account. They couldn't figure how to give me access. They wouldn't just remove my credit card from the account.

One month the charge became $10 so I called my bank and had them block AWS charges. It seemed the only way to deal with the fear that someday that would creep up and up.

Re: Please fix the AWS free tier before somebody gets hurt

#258

Post by tedk-42 » Wed, May 05, 2021, 5:42 AM UTC

paranoidrobot wrote: post #182

whoknowswhat11 wrote: post #139

Goodness - reading these statements makes me laugh - "sinister" "evil" "by design". Reality - very few folks want a hard billing limit. To stop charging you AWS would need to delete all your EBS and S3 volumes, stop all EC2 instances, release all public IP's, delete all AWS directories and the list goes on. The idea that AWS would build this giant data loss footgun into their system is ridiculous. Somewhere in AWS so…

AWS employs a whole lot of brilliant people, and they're an insanely profitable business unit. I'm sure those people, given the appropriate motivation and opportunity, could provide a solution (or several) that avoids foot-gun data loss at every turn. There's all sort of solutions they could go with. As an idea, perhaps some kind of functionality within the billing side of things: They could, perhaps, decide to just…

It's a good lesson in cloud computing to know how much you're spending on services.

Usually if you're a single person and you leave an expensive product running unknowingly, AWS support have been kind and refund you the money and take the loss as 'customer made an error' type transaction.

It's not like they hold the debt over your head for life.

I think the article is a bit extreme to highlight the person's reaction, it's pretty easy bait for most people to swallow. Realistically though, if you can't afford $20, how on earth did you afford a credit card to sign up for AWS in the first place?

There are people comparing this person's response with the Robinhood investor that lost money and took their life. The real issue here is a mental health problem and better awareness around that - AWS is a cheap scapegoat we can all blame to ignore the real issue at hand...

Re: Please fix the AWS free tier before somebody gets hurt

#259

Post by krainboltgreene » Wed, May 05, 2021, 5:46 AM UTC

kaishiro wrote: post #250

jiggawatts wrote: post #136

Everyone: PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits. As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits. There's a difference between "unable" and "unwilling". The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't s…

"Every other cloud does this! They all have non-production subscription types with hard spending limits." This is simply not true, to the best of my knowledge. GCP has budget limits, which can send alerts , but outside of manually catching a pubsub and shutting down billing on your own, there are no configurable hard billing limits. I'm unsure what to make of the rest of your rant.

The budget limits are delayed hard billing limits. Everything I've seen/heard suggests while you can go over, they absolutely don't charge the overage.

Re: Please fix the AWS free tier before somebody gets hurt

#260

Post by kaishiro » Wed, May 05, 2021, 5:50 AM UTC

krainboltgreene wrote: post #259

kaishiro wrote: post #250

Earlier quoted context omitted.

"Every other cloud does this! They all have non-production subscription types with hard spending limits." This is simply not true, to the best of my knowledge. GCP has budget limits, which can send alerts , but outside of manually catching a pubsub and shutting down billing on your own, there are no configurable hard billing limits. I'm unsure what to make of the rest of your rant.

The budget limits are delayed hard billing limits. Everything I've seen/heard suggests while you can go over, they absolutely don't charge the overage.

This is simply false.

We go over our budget "limits" literally every month. We do this because they are simply alerts, nothing more. We have separate alerts set up for 25, 50, 75, and 100% of our "limit" so that we can track variable usage and catch spikes early.

We, of course, pay our full bill every month - over the "limit".