I have a feeling I will be downvoted for this, but... Have the people posting these horror stories never heard of billing alerts?
There is also WAF.
Serverless Horrors
181–190 of 503 posts
Re: Serverless Horrors
#182look mom & dad, I am famous!
Re: Serverless Horrors
#183I guess I'm missing something, why is this 'serverless' horrors? If anything it seems to specifically be serverful horrors.
"Serverless" is just marketing-speak for "somebody else's server".
Re: Serverless Horrors
#184When I was learning to program through a bootcamp I spun up an elastic beanstalk instance that was free but required a credit card to prove your identity. No problem that makes sense - it's an easy way to prove authentication as a bot can't spam a credit card (or else it would be financial fraud and most likely a felony). Amazon then charged me one hundred thousand dollars as the server was hit by bot spam. I had the…
I’ve never trusted AWS with personal work for exactly this reason. If I want to spend $20 on a personal project I should be able to put a cap on that directly, not wake up to a $100k bill and go through the stress of hoping it might be forgiven.
Re: Serverless Horrors
#185When I was learning to program through a bootcamp I spun up an elastic beanstalk instance that was free but required a credit card to prove your identity. No problem that makes sense - it's an easy way to prove authentication as a bot can't spam a credit card (or else it would be financial fraud and most likely a felony). Amazon then charged me one hundred thousand dollars as the server was hit by bot spam. I had the…
I would never use a cloud service that doesn't let me set a hard cap for any service. Not just an alert. A hard cap.
Re: Serverless Horrors
#186Earlier quoted context omitted.
Its not expensive for them, its expensive for their customers. If you went over your spending limit and they deleted all your shit, people would be absolutely apoplectic. Instead they make you file a relatively painless ticket and explain why you accidentally went over what you wanted to spend. This is an engineering trade-off they made to make things less painful for their customers.
Why would I apoplectic at Amazon if I set “turn my shit off after it has accrued $10 in charges” to TRUE and they actually followed what I asked them to do?
Is it a serious question? Because then I could have you shutdown just by posting a call to ddos with a link to your search form on an anime image board.
Re: Serverless Horrors
#187I remember at the beginning of the serverless hype how they said it was great because it automatically scaled as big as you need it. Given how sudden and massive these "scaling spikes" can be, I would much rather deal with a death-hugged VPS than a $100k bill.
Plus the VPS is just so much faster in most cases.
Re: Serverless Horrors
#188The assignment of blame for misconfigured cloud infra or DOS attacks is so interesting to me. There don't seem to be many principles at play, it's all fluid and contingent.
Customers demand frictionless tools for automatically spinning up a bunch of real-world hardware. If you put this in the hands of inexperienced people, they will mess up and end up with huge bills, and you take a reputational hit for demanding thousands of dollars from the little guy. If you decide to vet potential customers ahead of time to make sure they're not so incompetent, then you get a reputation as a gatekeeper with no respect for the little guy who's just trying to hustle and build.
I always enjoy playing at the boundaries in these thought experiments. If I run up a surprise $10k bill, how do we determine what I "really should owe" in some cosmic sense? Does it matter if I misconfigured something? What if my code was really bad, and I could have accomplished the same things with 10% of the spend?
Does it matter who the provider is, or should that not matter to the customer in terms of making things right? For example, do you get to demand payment on my $10k surprise bill because you are a small team selling me a PDF generation API, even if you would ask AWS to waive your own $10k mistake?
Re: Serverless Horrors
#189Earlier quoted context omitted.
Amazon refunded you and you hate them for it? I think one of the reasons I appreciate AWS so much is that any time there has been snafu that led to a huge bill like this they've made it pretty painless to get a refund- just like you experienced.
If it is a "free tier", Amazon should halt the application when it exceeds quota. Moving the account to a paid tier and charging $100k is not the right thing to do.
Good news! This is exactly how the free tier works now.
Re: Serverless Horrors
#190Don’t most of these services have config options to protect against doing this? I haven’t used most of these services but it running up a bill during traffic spikes but not going down seems like it’s working as intended?
I know AWS in particular does not because they do not increment the bill for every request. I don't know exactly how they calculate billing, but based on what I do know about it, I imagine it as a MapReduce job that runs on Lambda logs every so often to calculate what to bill each user for the preceding time interval.
That billing strategy makes it impossible to prevent cost overruns because by the time the system knows your account exceeded the budget you set, the system has already given out $20k worth of gigabyte-seconds of RAM to serve requests.
I think most other serverless providers work the same way. In practice, you would prevent such high traffic spikes with rate limiting in your AWS API Gateway or equivalent to limit the amount of cost you could accumulate in the time it takes you to receive a notification and decide on a course of action.