Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 348 posts

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#11

Post by bombcar »

Waterluvian wrote: post #10
em1sar wrote: post #7

It's really difficult for AWS or any other serverless provider for that matter, to achieve a kind of "bulletproof and safe user experience" across different offerings that encompasses everything that has to do with billing/monitoring/alerting and then also cover all kinds of potential customer scenarios (like the function calling itself, as one example). For example, it's totally understandable that the alarms can be…

What’s the case for not implementing an optional “shut down all my services at $spend and stay shut down until I intervene” ?

Honestly? Many people would enable it, forget about it, and footgun themselves on the other side.

Perhaps AWS should have "personal/developer" accounts that have this enabled by default and continually warn you about it, whereas "company/enterprise" don't have them.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#12

Post by IshKebab »

em1sar wrote: post #7

It's really difficult for AWS or any other serverless provider for that matter, to achieve a kind of "bulletproof and safe user experience" across different offerings that encompasses everything that has to do with billing/monitoring/alerting and then also cover all kinds of potential customer scenarios (like the function calling itself, as one example). For example, it's totally understandable that the alarms can be…

It's not really difficult. They just need a way to set hard spending limits. Probably on by default.

Unless you're a big company, "we stopped your function in the middle of the night" is a whole lot better than "we ran your function all night and you owe us $4k".

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#13

Post by JamesSwift »

I had an ECS cron job hang one time, so instead of a 30 second compute charge it ended up being almost a full month of continuous runtime. My usual $30 bill was $800, and the estimated charges for the next month based on 2 days of use was $1300. I didn't have a billing alert setup (definitely setup a billing alert!).

It could not have been easier to get AWS to remove the charge. A quick email to support with a brief explanation and it was immediately accepted. The hardest part was that they wanted a very specific request for how much I was asking to be refunded. So I had to go back and calculate my average costs per service and compare that to the charged costs. After that it was immediately refunded.

They aren't just handing these things out though. They made me read and acknowledge I had read their service agreements and basically swear that I know what happened and it won't happen again. Really painless process overall, all things considered.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#15

Post by thdxr »

ralusek wrote: post #9

One of my biggest fears. What's to prevent trolls and competitors from just spamming your endpoints in a loop? How do people using pay-per-use infra deal with these problems? I really want to use Lambda for public endpoints but it just scares me.

There are a lot of tools in AWS to deal with this. Api Gateway Rate Limiting, WAF, etc.

That said it does take a level of awareness to set these things up. Some of what we try to do at SST is turn on these for you automatically so you're not being punished for not knowing something

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#16

Post by mirker »

bombcar wrote: post #11
Waterluvian wrote: post #10

Earlier quoted context omitted.

What’s the case for not implementing an optional “shut down all my services at $spend and stay shut down until I intervene” ?

Honestly? Many people would enable it, forget about it, and footgun themselves on the other side. Perhaps AWS should have "personal/developer" accounts that have this enabled by default and continually warn you about it, whereas "company/enterprise" don't have them.

I’d think if your rate of spending is >$50/hour then that’s nearly-always a bug. The only reason this conversation is taking place is because serverless “infinitely scales”. Autoscaling physical instances has a max limit for similar reasons.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#17

Post by upupandup »

reminds me when I set up an S3 triggered lambda function that also wrote into the same directory. What ensued was millions of files and folders generated recursively.

Fortunately, AWS was kind enough to reverse the billing. Had this been Google Cloud, I would've gotten the cold shoulder and a low key threat that if I reverse the transaction I would lose access to my other paid Google products outside GCP :/

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#18

Post by panki27 »

drdrek wrote: post #8

AWS may promote the technologies as prototype friendly but at the end of the day its built to be enterprise grade production tool. A company will not even bother with a 4000$ mistake, its just the price of doing business so there is little incentive to address these types of problems. Playing around with AWS for side projects is like using a chainsaw, it can really accelerate your work but if you are going to make a…

Chainsaw is the perfect metaphor for this, well done.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#20

Post by capableweb »

Off-topic maybe, but I'm very curious as it's hard to find hard numbers:

> my usual bill is $200/month

How many req/second are you serving? What kind of things are happening?

It seems like the bills are outrageously expensive when it comes to various cloud services, as I'm personally hosting a service that does between 10-100 req/second on average during a month, and my monthly bill end up being closer to $40/month, including traffic and everything. I'm running a database on the same server, and 20% of the requests writes stuff both to disk and to the database, while the rest just reads from DB or disk.

The whole setup took around 5 hours to setup on one day, and have been running flawlessly from day one, we haven't had to migrate servers yet after ~6 months of production usage. Probably one day we're gonna have to upgrade the server to a $80/month one, but that's a one-time thing and our revenue easily covers that.

Post reply on HN