Live data from Hacker News

Serverless Horrors

serverlesshorrors.com

191–200 of 503 posts

Re: Serverless Horrors

#191

Earlier quoted context omitted.

Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.

If it is a "free tier", Amazon should halt the application when it exceeds quota. Moving the account to a paid tier and charging $100k is not the right thing to do.

I agree, but I could also see how someone would complain about that: “Our e-commerce site was taken down by Amazon right on our biggest day of the year. They should have just moved us up to the next tier.”

Re: Serverless Horrors

#192

Earlier quoted context omitted.

Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.

If it is a "free tier", Amazon should halt the application when it exceeds quota. Moving the account to a paid tier and charging $100k is not the right thing to do.

stop putting stuff on the internet you don't understand.

Re: Serverless Horrors

#193
post #114

Earlier quoted context omitted.

These aren’t limits though, they are just budget notifications. What would be helpful, would be if when you set up your account there was a default limit – as in an actual limit, where all projects stop working once you go over it - of some sane amount like $5 or $50 or even $500. I have a handful of toy projects on AWS and Google cloud. On both I have budgets set up at $1 and $10, with notifications at 10% 50% and 9…

I feel that the likely answer here is that instrumenting real-time spending limit monitoring and cut-off at GCP/AWS scale is Complicated/Expensive to do, so they choose to not do it. I suppose you could bake the limits into each service at deploy time, but that's still a lot of code to write to provide a good experience to a customer who is trying to not pay you money. Not saying this is a good thing, but this feels…

Pass a law requiring cloud compute providers to accept a maximum user budget and be unable to charge more than that, and see how quickly the big cloud providers figure it out.

Re: Serverless Horrors

#194
post #83

When I was learning to program through a bootcamp I spun up an elastic beanstalk instance that was free but required a credit card to prove your identity. No problem that makes sense - it's an easy way to prove authentication as a bot can't spam a credit card (or else it would be financial fraud and most likely a felony). Amazon then charged me one hundred thousand dollars as the server was hit by bot spam. I had the…

This is an example of why cloud hosting is so scary. Yes, Amazon, and I assume Azure and Google's cloud and others, "usually" refund the money. But I don't want to be forced into bankruptcy because my five visitor a week demo project suddenly becomes the target of a DDOS for no reason at all and the hosting company decides this isn't a "usually" so please send the wire transfer.

If you sign up for electrical service for your house, and your shithead neighbor taps your line to power his array of grow lamps and crypto mining rigs, the power company will happily charge you thousands of dollars, and you will need a police report and traverse many layers of customer service hell to get a refund. If you sign up for water service and a tree root cracks your pipe, the water company will happily charge you thousands of dollars for the leaked water, and will then proceed to mandate that you to fix the broken pipe at your own expense for a couple tens of thousands more; and yes, that may well bankrupt you, water company don't care. So why do you expect different treatment from a computing utility provider?

Re: Serverless Horrors

#195
post #134

Earlier quoted context omitted.

> CloudFront's ~$100/TB after 1TB. I had to double-check because that sounds hilariously wrong. I can't find it anywhere in the pricing. It's at most 0.08/TB. Am I missing something?

You're missing the unit, it's $0.085 per GB, not TB, and that's only for NA/EU traffic. I rounded up a bit from that number because other regions cost more, plus you get billed a flat amount for each request as well. They do offer progressively cheaper rates as you use more bandwidth each month, but that doesn't have much impact until you're already spending eye watering amounts of money.

Oh, yeah, egg on my face. They only put the unit of measurement at the top, and then talk about TB, so it's a bit deceptive. In retrospect, I was stupid to imagine 0.085/TB made any sense.

Re: Serverless Horrors

#196
post #174

Earlier quoted context omitted.

Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.

Amazon is irresponsible when they let people sign up for a unlimited credit. At minimum they should provide hard billing caps.

putting stuff on the internet is dangerous. if you're not prepared to secure public endpoints stop creating them.

Re: Serverless Horrors

#197

Earlier quoted context omitted.

I don't care if it is expensive for them. I'm not running their business, I'm their customer - it is inconvenient for me. And frankly any pay-as-you-go scheme should be regulated to have maximum spending limit setting. Not only in IT.

Its not expensive for them, its expensive for their customers. If you went over your spending limit and they deleted all your shit, people would be absolutely apoplectic. Instead they make you file a relatively painless ticket and explain why you accidentally went over what you wanted to spend. This is an engineering trade-off they made to make things less painful for their customers.

There is a huge difference between deleting data and stopping running services.

You're right in that there's a few services that expose this complexity directly, the ones where you're paying for actual storage, but this is just complex, not impossible.

For one thing, storage costs are almost always static for the period, they don't scale to infinite in the same way.

Re: Serverless Horrors

#198

The assignment of blame for misconfigured cloud infra or DOS attacks is so interesting to me. There don't seem to be many principles at play, it's all fluid and contingent. Customers demand frictionless tools for automatically spinning up a bunch of real-world hardware. If you put this in the hands of inexperienced people, they will mess up and end up with huge bills, and you take a reputational hit for demanding tho…

Maybe, but its a huge reason to use real servers instead of serverless.

Re: Serverless Horrors

#199
post #28

Are there any protections these days at the cloud provider level? Like setting a maximum budget for a certain service (EC2, Aurora?) because downtime is preferable to this?

Not _really_. AWS has a budget tool, but it doesn’t natively support shutting down services. Of course, you can ingest the alerts it sends any way you want, including feeding them into pipelines that disable services. There’s plenty of blueprints you can copy for this. More seriously - and this is a legitimate technical limitation - of course AWS doesn’t check each S3 request or Lambda invocation against your budget,…

> of course AWS doesn’t check each S3 request or Lambda invocation against your budget

If it can bill them per-invocation, why can't it also check against a budget? I don't expect it to be synchronous, but a lag of minutes to respond is still better than nothing. Can you even opt-in to shutting down services from the budget tool, or is that still something you have to script by hand from Cloudwatch alarms?

Re: Serverless Horrors

#200
post #131

Earlier quoted context omitted.

> By that logic, any technology that you can get certified in is too complicated? In IT, I am inclined to agree with that. In real engineering, it's sometimes necessary, especially dangerous technology and technology that people trust with their life

> dangerous technology and technology that people trust with their life Software runs on so many things we depend on IMO it also in many cases falls in the "dangerous technology" category. Non-hobby OSes, non-hobby web browsers, device drivers, software that runs critical infrastructure, software that runs on network equipment, software that handles personal data, --IMHO it would not be unreasonable to require formal…

If I go buy a TIG welder, use it without any training, leave it on and go get coffee, do I get to complain that I have to pay for a new house?
Post reply on HN