Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Serverless Horrors

serverlesshorrors.com

Post reply on HN
Print view View original

21–30 of 503 posts

Re: Serverless Horrors

#21

Post by tonyhart7 »

trcf22 wrote: post #2

After a quick check on Vercel stories, it seems all payments were discarded or mistakes in the first place. Does it really happen to really have to pay such a bill? Do you need to tweet about it to be reimbursed?

I mean if developer got charged with 100k, more often than not the bank would decline that first maybe if you didn't have that high credit limit

but what happen if this happen to corporate account and somewhere resource get leaked???

multi billions dollar company probably just shrug it off as opex and call it a day

Re: Serverless Horrors

#22

Post by paseante »

Yeah I also left my website hosted on Google Cloud because costs popped from everywhere, and there is basically no built-in functionality to limit them. So I didn't really slept relaxed (I actually slept great, but I hope you get the point) knowing that a bug could cost me... who knows how much. Actually, as the website of OP says, for spending control you have budget notifications and with that you can disable the billing for all the project altogether through some API call or something, I don't remember exactly, that is all there is. But still it looks like this functionality is just not there.

Re: Serverless Horrors

#23

Post by fishmicrowaver »

thousand_bats wrote: post #16
joshstrange wrote: post #4

I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. I scrolled aimlessly through the site ignoring most posts (bandwidth usage bills aren’t super interesting) but I did see this one: https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-c... About how you make unauth’d API calls to an s3 bucket you don’t own to run up the costs. That was a n…

> I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. Agreed about that. I was hired onto a team that inherited a large AWS Lambda backend and the opacity of the underlying platform (which is the value proposition of serverless!) has made it very painful when the going gets tough and you find bugs in your system down close to that layer (in our c…

I've had similar experiences with Azures services. Black boxes impossible to troubleshoot. Very unexpected behavior people aren't necessarily aware of when they initially spin these things up. For anything important I just accept the pain of deploying to kubernetes. Developers actually wind up preferring it in most cases with flux and devsoace.

Re: Serverless Horrors

#24

Post by mikepurvis »

thousand_bats wrote: post #16
joshstrange wrote: post #4

I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. I scrolled aimlessly through the site ignoring most posts (bandwidth usage bills aren’t super interesting) but I did see this one: https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-c... About how you make unauth’d API calls to an s3 bucket you don’t own to run up the costs. That was a n…

> I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. Agreed about that. I was hired onto a team that inherited a large AWS Lambda backend and the opacity of the underlying platform (which is the value proposition of serverless!) has made it very painful when the going gets tough and you find bugs in your system down close to that layer (in our c…

Is that what people do is test/develop primarily with local mocks of the services? I assumed it was more like you deploy mini copies of the app to individual instances namespaced to developer or feature branch, so everyone is working on something that actually fairly closely approximates prod just without the loading characteristics and btw you have to be online so no working on an airplane.

Re: Serverless Horrors

#25

Post by rwmj »

solatic wrote: post #14
dakiol wrote: post #10

> I had cloudflare in front of my stuff. Hacker found an uncached object and hit it 100M+ times. I stopped that and then they found my origin bucket and hit that directly. Pardon my ignorance, but isn’t that something that can happen to anyone? Uncached objects are not something as serious as leaving port 22 open with a weak password (or is it?). Also, aren’t S3 resources (like images) public so that anyone can hit t…

No. Your buckets should be private, with a security rule that they can only be accessed by your CDN provider, precisely to force the CDN to be used.

Why isn't that the default?

I'm glad I use a Hetzner VPS. I pay about EUR 5 monthly, and never have to worry about unexpected bills.

Re: Serverless Horrors

#26

Post by franktankbank »

sherburt3 wrote: post #7
joshstrange wrote: post #4

I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. I scrolled aimlessly through the site ignoring most posts (bandwidth usage bills aren’t super interesting) but I did see this one: https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-c... About how you make unauth’d API calls to an s3 bucket you don’t own to run up the costs. That was a n…

I believe they changed that shortly after that blog post went viral: https://aws.amazon.com/about-aws/whats-new/2024/08/amazon-s3...

Seems an interesting oversight. I can just imagine the roundtable, uhh guys who do we charge for 403? Who can we charge? But what if people hit random buckets as an attack? Great!

Re: Serverless Horrors

#27

Post by tonkinai »

mikepurvis wrote: post #24
thousand_bats wrote: post #16

Earlier quoted context omitted.

> I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. Agreed about that. I was hired onto a team that inherited a large AWS Lambda backend and the opacity of the underlying platform (which is the value proposition of serverless!) has made it very painful when the going gets tough and you find bugs in your system down close to that layer (in our c…

Is that what people do is test/develop primarily with local mocks of the services? I assumed it was more like you deploy mini copies of the app to individual instances namespaced to developer or feature branch, so everyone is working on something that actually fairly closely approximates prod just without the loading characteristics and btw you have to be online so no working on an airplane.

Mocks usually don’t line up with how things run in prod. Most teams just make small branch or dev environments, or test in staging. Once you hit odd bugs, serverless stops feeling simple and just turns into a headache.

Re: Serverless Horrors

#28

Post by EGreg »

Are there any protections these days at the cloud provider level?

Like setting a maximum budget for a certain service (EC2, Aurora?) because downtime is preferable to this?

Re: Serverless Horrors

#29

Post by Havoc »

Putting any sort of pay per use product onto the open internet has always struck me as insane. Especially with scaling enabled.

At least stick a rate limited product in front of it to control the bleed. (And check whether the rate limit product is in itself pay per use...GCP looking at you)

Re: Serverless Horrors

#30

Post by icedchai »

joshstrange wrote: post #4

I thought this would be about the horrors of hosting/developing/debugging on “Serverless” but it’s about pricing over-runs. I scrolled aimlessly through the site ignoring most posts (bandwidth usage bills aren’t super interesting) but I did see this one: https://medium.com/@maciej.pocwierz/how-an-empty-s3-bucket-c... About how you make unauth’d API calls to an s3 bucket you don’t own to run up the costs. That was a n…

Same, I was hoping for tales of woe and cloud lock-in, of being forced to use Lambda and Dynamo for something that could easily run on a $20/month VPS with sqlite.
Post reply on HN