Earlier quoted context omitted.
Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.
If it is a "free tier", Amazon should halt the application when it exceeds quota. Moving the account to a paid tier and charging $100k is not the right thing to do.
Serverless Horrors
191–200 of 503 posts
Re: Serverless Horrors
#192Earlier quoted context omitted.
Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.
If it is a "free tier", Amazon should halt the application when it exceeds quota. Moving the account to a paid tier and charging $100k is not the right thing to do.
Re: Serverless Horrors
#193Earlier quoted context omitted.
These aren’t limits though, they are just budget notifications. What would be helpful, would be if when you set up your account there was a default limit – as in an actual limit, where all projects stop working once you go over it - of some sane amount like $5 or $50 or even $500. I have a handful of toy projects on AWS and Google cloud. On both I have budgets set up at $1 and $10, with notifications at 10% 50% and 9…
I feel that the likely answer here is that instrumenting real-time spending limit monitoring and cut-off at GCP/AWS scale is Complicated/Expensive to do, so they choose to not do it. I suppose you could bake the limits into each service at deploy time, but that's still a lot of code to write to provide a good experience to a customer who is trying to not pay you money. Not saying this is a good thing, but this feels…
Re: Serverless Horrors
#194When I was learning to program through a bootcamp I spun up an elastic beanstalk instance that was free but required a credit card to prove your identity. No problem that makes sense - it's an easy way to prove authentication as a bot can't spam a credit card (or else it would be financial fraud and most likely a felony). Amazon then charged me one hundred thousand dollars as the server was hit by bot spam. I had the…
This is an example of why cloud hosting is so scary. Yes, Amazon, and I assume Azure and Google's cloud and others, "usually" refund the money. But I don't want to be forced into bankruptcy because my five visitor a week demo project suddenly becomes the target of a DDOS for no reason at all and the hosting company decides this isn't a "usually" so please send the wire transfer.
Re: Serverless Horrors
#195Earlier quoted context omitted.
> CloudFront's ~$100/TB after 1TB. I had to double-check because that sounds hilariously wrong. I can't find it anywhere in the pricing. It's at most 0.08/TB. Am I missing something?
You're missing the unit, it's $0.085 per GB, not TB, and that's only for NA/EU traffic. I rounded up a bit from that number because other regions cost more, plus you get billed a flat amount for each request as well. They do offer progressively cheaper rates as you use more bandwidth each month, but that doesn't have much impact until you're already spending eye watering amounts of money.
Re: Serverless Horrors
#196Earlier quoted context omitted.
Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.
Amazon is irresponsible when they let people sign up for a unlimited credit. At minimum they should provide hard billing caps.
Re: Serverless Horrors
#197Earlier quoted context omitted.
I don't care if it is expensive for them. I'm not running their business, I'm their customer - it is inconvenient for me. And frankly any pay-as-you-go scheme should be regulated to have maximum spending limit setting. Not only in IT.
Its not expensive for them, its expensive for their customers. If you went over your spending limit and they deleted all your shit, people would be absolutely apoplectic. Instead they make you file a relatively painless ticket and explain why you accidentally went over what you wanted to spend. This is an engineering trade-off they made to make things less painful for their customers.
You're right in that there's a few services that expose this complexity directly, the ones where you're paying for actual storage, but this is just complex, not impossible.
For one thing, storage costs are almost always static for the period, they don't scale to infinite in the same way.
Re: Serverless Horrors
#198The assignment of blame for misconfigured cloud infra or DOS attacks is so interesting to me. There don't seem to be many principles at play, it's all fluid and contingent. Customers demand frictionless tools for automatically spinning up a bunch of real-world hardware. If you put this in the hands of inexperienced people, they will mess up and end up with huge bills, and you take a reputational hit for demanding tho…
Re: Serverless Horrors
#199Are there any protections these days at the cloud provider level? Like setting a maximum budget for a certain service (EC2, Aurora?) because downtime is preferable to this?
Not _really_. AWS has a budget tool, but it doesn’t natively support shutting down services. Of course, you can ingest the alerts it sends any way you want, including feeding them into pipelines that disable services. There’s plenty of blueprints you can copy for this. More seriously - and this is a legitimate technical limitation - of course AWS doesn’t check each S3 request or Lambda invocation against your budget,…
If it can bill them per-invocation, why can't it also check against a budget? I don't expect it to be synchronous, but a lag of minutes to respond is still better than nothing. Can you even opt-in to shutting down services from the budget tool, or is that still something you have to script by hand from Cloudwatch alarms?
Re: Serverless Horrors
#200Earlier quoted context omitted.
> By that logic, any technology that you can get certified in is too complicated? In IT, I am inclined to agree with that. In real engineering, it's sometimes necessary, especially dangerous technology and technology that people trust with their life
> dangerous technology and technology that people trust with their life Software runs on so many things we depend on IMO it also in many cases falls in the "dangerous technology" category. Non-hobby OSes, non-hobby web browsers, device drivers, software that runs critical infrastructure, software that runs on network equipment, software that handles personal data, --IMHO it would not be unreasonable to require formal…