Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Please fix the AWS free tier before somebody gets hurt

cloudirregular.substack.com

Post reply on HN
Print view View original

241–250 of 457 posts

Re: Please fix the AWS free tier before somebody gets hurt

#241

Post by KirillPanov »

NathanKP wrote: post #106
NullPrefix wrote: post #28

Earlier quoted context omitted.

IIRC the excuse is that billing is a separate department and they count all the usage dollars way after you done using it, not in realtime. You would still be able to go over your limit and then who should foot the bill? Realtime counting would be too difficult to figure out, our brightest minds are busy figuring out engagement metrics.

This is my personal opinion, though I do work at AWS: It's not that real time counting is difficult, it is that the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical. There is a reason why banks and financial institutions generally do batch processing in the off peak hours when electricity is cheaper and there is less demand for the compute…

> the amount of compute resources and electricity that would be needed to power real time billing at AWS scale would be astronomical

You don't have to bill in real time.

You just have to provision funding for every resource except network bandwidth.

Customer sets a monthly spend limit. Every time they start up an instance, create a volume, allocate an IP, or do anything else that costs money, you subtract the monthly cost of that new thing from their spend limit. If the spend limit would go negative, you refuse to create the new resource.

If the spend limit is still positive, the remaining amount is divided by the number of seconds remaining in the month times the bandwidth cost. The result becomes that customer's network throughput limit. Update that setting in your routers (qdisc in Linux) as part of the API call that allocated a resource. If you claim your routers don't have a limit like this I call shenanighans.

This should work perfectly for one region.

There's probably a way to generalize it to multiple regions, but I'm sure most small/medium customers would be happy enough to have a budget for each region. They'd probably set most regions' budget to zero and just worry about one or two.

The web UI probably would need to be updated to show the customer "here is what your bandwidth limit for the rest of the month will be if you proceed; are you sure?". JSON APIs can return this value when invoked in dry-run mode.

Re: Please fix the AWS free tier before somebody gets hurt

#242

Post by czgs2woh4ue3 »

Retric wrote: post #170
czgs2woh4ue3 wrote: post #156

Earlier quoted context omitted.

I also work at AWS (nowhere near billing), so the usual disclaimers apply, but: I actually have no idea if billing is real-time or not? I think it's mostly batch, but the records aren't necessarily, though they may be aggregated a bit. The general point in this thread certainly holds: our systems provide service first, bill second, and that by throwing a record over the wall to some other internal system. It's not un…

“Small, cheap things like "a single DynamoDB query" or "a byte of bandwidth" are often also high-volume, and you don't want to introduce billing checks to critical paths for reliability / latency / cost reasons” That’s hardly necessary. Let’s suppose you have some service that costs 1 cent every 1,000 queries. If you’re billing it then you need to be keeping track of it and incrementing some counter somewhere. If old…

That counter may well not exist outside of billing for longer than it takes to batch some records together. It will need to be shared and synchronized with the rest of the fleet, the other fleets in the availability zone, the other zones in the region, the other regions, and every other service in AWS. There are strict rules about crossing these boundaries for sake of failure isolation.

As an amusing extra sticking point, your service has no idea how much it actually costs, because that's calculated by billing- the rates may vary from request to request or from customer to customer.

Without spending way too long thinking about it, the complexity in figuring out exactly when to stop is significant enough that it probably cannot practically be done in the critical path of these kinds of high-volume systems, hence the reactive approach being more plausible to me.

I don't know what kinds of problems AT&T has, but at the risk of saying dumb things about an industry I know next to nothing about, your phone is only attached to one tower at a time, and that probably helps a bit. And I'm not sure when it wouldn't be simpler and just as good for them to also react after the fact, anyway.

Re: Please fix the AWS free tier before somebody gets hurt

#243

Post by apatters »

artichokeheart wrote: post #24

AWS employs cost obfuscation by design otherwise the default view when you open the console would show you all of your current active services. Not only is that not the case, a single screen to show you all of your current active services doesn't exist. You need to take a deep dive into cost explorer (assuming you have access in corporate land) and try to decipher in what that all means.

Until AWS fixes this the best thing to do is just use their service as little as possible. There are plenty of other cloud providers out there these days which don't employ this hostile practice.

I closed an AWS account for this reason just a few days ago, we hadn't used it for a while but there was no clear way to remove our credit card so it felt like a risk just to have it open, what if a developer logs in to mess around and accidentally flips some switch that smacks us with a charge of a couple grand... unlikely, sure, but the fact that it's even possible is terrible system design. Better to just nuke the account and move on to other cloud providers that don't make it harder for me to sleep at night.

Re: Please fix the AWS free tier before somebody gets hurt

#244

Post by KirillPanov »

Retric wrote: post #170
czgs2woh4ue3 wrote: post #156

Earlier quoted context omitted.

I also work at AWS (nowhere near billing), so the usual disclaimers apply, but: I actually have no idea if billing is real-time or not? I think it's mostly batch, but the records aren't necessarily, though they may be aggregated a bit. The general point in this thread certainly holds: our systems provide service first, bill second, and that by throwing a record over the wall to some other internal system. It's not un…

“Small, cheap things like "a single DynamoDB query" or "a byte of bandwidth" are often also high-volume, and you don't want to introduce billing checks to critical paths for reliability / latency / cost reasons” That’s hardly necessary. Let’s suppose you have some service that costs 1 cent every 1,000 queries. If you’re billing it then you need to be keeping track of it and incrementing some counter somewhere. If old…

> PS: Phone companies can pull this stuff off successfully for millions of cellphones. If you’re arguing keeping up with AT&T is to hard, you have serious organizational problems.

To be fair, AT&T in particular does prepaid shutoffs on a pretty coarse granularity, I think it's like 15-minute intervals.

I know this because for a while I had to use a prepaid LTE modem as my primary internet connection. You can use as much bandwidth as you want for the remainder of the 15-minute interval in which you exceed what you've paid for -- then they shut you off.

I once managed (by accident) to get 3GB out of a 2GB plan purchase because of this.

Of course that free 1GB was only free because I consumed all of it in the 14.9 minute time period preceding NO CARRIER.

Re: Please fix the AWS free tier before somebody gets hurt

#245

Post by rk06 »

dinkleberg wrote: post #164
jiggawatts wrote: post #136

Everyone: PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits. As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits. There's a difference between "unable" and "unwilling". The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't s…

Which clouds do this? I work quite a bit with AWS and GCP, and Azure to a lesser extent, and I've yet to see a spending limit on any of them. I'd love to be proven wrong though.

Azure gives you credits to spend. And after you have exhausted your credits, then they don't charge more.

If you have Visual Studio professional or enterprise license, then you can avail recurring credits, which are useful for learning

Re: Please fix the AWS free tier before somebody gets hurt

#246

Post by hardwaregeek »

singlow wrote: post #235
artichokeheart wrote: post #24

AWS employs cost obfuscation by design otherwise the default view when you open the console would show you all of your current active services. Not only is that not the case, a single screen to show you all of your current active services doesn't exist. You need to take a deep dive into cost explorer (assuming you have access in corporate land) and try to decipher in what that all means.

I don't understand why so many people want to use AWS for such small budgets. At that scale wouldn't it be easier to just build everything out of a few VMs at a cheap service? AWS is awesome when you have a large number of resources, that are created programmatically and reproducibly, with redundancy and duplicate environments. The budgeting tools are really amazing at letting you categorize your costs and create app…

Linux server administration requires a certain amount of know-how and determination. I can't tell you how many times I had to rebuild a DigitalOcean server because I messed something up and wanted to start with a clean slate.

A lot of hackathons, workshops and courses ask you to use AWS these days. Whether it's to run machine learning instances, win a sponsor prize or learn how to use Lambda, students are often encouraged to learn one of the major cloud providers.

Also it's a resume boost. It's another buzzword you can add to your resume.

Re: Please fix the AWS free tier before somebody gets hurt

#247

Post by 0xmohit »

This appears to be by design. People have been demanding a budget capping service [0] for a decade now. AWS would continue to bill for compromised accounts [1]. AWS "free" offerings can cost a lot more than you think [2].

[0] https://news.ycombinator.com/item?id=3356987

[1] https://news.ycombinator.com/item?id=20045615

[2] https://news.ycombinator.com/item?id=22988647

Re: Please fix the AWS free tier before somebody gets hurt

#248

Post by throwawaaarrgh »

PeterisP wrote: post #76
throwawaaarrgh wrote: post #16

Oh come on. We all know that the 'accidental revenue' from the way Free Tier is set up probably makes up a cool 2 million or more annually. Plenty to justify its continued abuse of naive students. Why would they walk away from that cash? The only people they're pissing off is people who aren't using AWS anyway.

AWS annual revenue is $50+ billion. ( https://www.zdnet.com/article/aws-run-rate-hits-54-billion-a... ) Anything that they can get from "naive students" and developers who don't notice small recurring charges is so utterly insignificant that it can't justify any decision whatsoever. If the public relations aspects of it cause even a 0.01% change in AWS growth, that's already $5 million of lost revenue; if you can ass…

If bad PR affected their business, they might have something to consider. But this will never affect their growth. They're not going to lose a single solitary sale that isn't already accounted for by their standard profit model. The only bad PR that would affect their business is if their actual reputation as a service provider were tarnished. Charging people money for a service they used does not tarnish AWS's reputation. It's their whole business model.

When you stay in a hotel room longer than you're supposed to and they charge you for another half day, or you eat food from the mini fridge, or buy a pay-per-view movie, etc, you could claim ignorance, and create stories about how terrible it is that people get charged for these things unknowingly. But Holiday Inn's bookings are still not going to take a noticeable hit.

Moreover, they don't even need to offer a Free Tier to get people to use AWS. They're AWS. They're the gold standard. It's like saying IBM would need a Free Tier (back when IBM wasn't trash). It wasn't a question of whether you should use IBM, it was whether you could afford it.

Re: Please fix the AWS free tier before somebody gets hurt

#249

Post by davidgatti »

jiggawatts wrote: post #136

Everyone: PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits. As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits. There's a difference between "unable" and "unwilling". The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't s…

Hmm not sure where the problem is. With IAM you can create a policy to limit the user to only use certain services, deploy only the cheapest type of instance type, limit the region, the time of day etc. All of this and more is built in to IAM by default. You can even create a CloudFormation as a internal product this way you can limit even more what people do, and budget for that. And on top of that you can make a Lambda that is triggered every 1h to shut down unused resources or send a warning to shut it down, if not, it will go down the next day.

The possibilities are endless. I personally don't see the IBM comparison.

Re: Please fix the AWS free tier before somebody gets hurt

#250

Post by kaishiro »

jiggawatts wrote: post #136

Everyone: PLEASE stop making the argument that it's "too hard" or even "impossible" to implement spending limits. As the article points out: Every other cloud does this! They all have non-production subscription types with hard spending limits. There's a difference between "unable" and "unwilling". The people that refuse to understand the difference are the same people that don't understand that "unsupported" isn't s…

"Every other cloud does this! They all have non-production subscription types with hard spending limits."

This is simply not true, to the best of my knowledge. GCP has budget limits, which can send alerts, but outside of manually catching a pubsub and shutting down billing on your own, there are no configurable hard billing limits.

I'm unsure what to make of the rest of your rant.

Post reply on HN