Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 348 posts

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#91

Post by ctvo »

mrozbarry wrote: post #29

At work, my development team is contracted with a company that uses AWS, and for better or worse, we have also become the devops team. We have been burned by AWS before, and we have a rule of thumb: if you are deploying new functionality/service communication, after deploy, monitor for 10-15 minutes, with a wide enough window to see if there is a noticable/unexpected change from before the deploy. It always feels lik…

> if you are deploying new functionality/service communication, after deploy, monitor for 10-15 minutes, with a wide enough window to see if there is a noticable/unexpected change from before the deploy. It always feels like wasted/burned time, but better to waste time than money.

... have you considered automating this? Alarms are pretty straight forward across all cloud platforms. Since you're using AWS: CloudWatch has anomaly detection. I haven't used it personally but perhaps it's worthwhile to look into: https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitori...

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#92

Post by devmor »

salawat wrote: post #81
Keyframe wrote: post #75

Earlier quoted context omitted.

Yeah, but how will you scale up your multi-dozen-users-at-the-same-time app easily then?

Buy more servers, loadbalance, and don't take the short painful route architecture-wise? It isn't really that hard.

There is a point where that becomes exponentially prohibitively expensive.

At a former startup where I worked as senior engineer, that was our original approach. Then one weekend we tripled our userbase and horizontally scaling required massive changes in database architecture, sharding solutions, etc.

"Just buy more servers and load balance" is the short painful route. Carefully planning out and taking advantage of scalable architecture that can be provided to you less expensively because it too, runs at scale is the hard method. The fact that its easy to shoot yourself in the foot with it doesn't make it the easy route.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#93

Post by maccard »

logifail wrote: post #67
cjbgkagh wrote: post #58

Earlier quoted context omitted.

Just like in real life, if you run out of money you have to stop doing things.

> Just like in real life, if you run out of money you have to stop doing things In real life when you hit your card's limit, your transactions get declined. Straight away. I had this last month in a supermarket after my (personal) checking account didn't have enough money to cover my purchase, I'd completely forgotten to transfer money from my business account. My bank wasn't prepared to let my account go overdrawn,…

> In real life when you hit your card's limit, your transactions get declined. Straight away.

Except for when it doesn't. I've got two primary current accounts, one with a "legacy" bank in the UK and one with a modern bank. The legacy bank is happy to let me go into an unplanned overdraft, and charge me for the privilege of doing so.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#94

Post by maccard »

ryanSrich wrote: post #82

I wish cloud providers had a nuclear option. Like “if my monthly spend hits $X, then just stop everything immediately”. Often these billing issues happen on little hobby projects and things that the owner would clearly be fine taking offline to avoid thousands in fees.

> then just stop everything immediately

What does stop everything immediately mean for things that aren't compute? Backups and storage, for example.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#95

Post by swatcoder »

Mo3 wrote: post #80
sirwitti wrote: post #74

I'm very much on the boring technology side of things with respect to hosting. 40€ / month gets you a very powerful dedicated server that can easily handle millions of requests per day and performs incredibly well and can be managed easily. If you also use containers you even get quite a bit of flexibility and agility. To be honest I don't really understand the sentiment that developers can get away with not knowing…

You're going to understand where the cool kids come from once your single dedicated server goes down or can't handle the load any more. As soon as you try to scale horizontally or become highly available and start to think about how to do it you end up falling into the same rabbit hole. > If you also use containers you even get quite a bit of flexibility and agility. Yeah... and then the only difference is between a…

I'm not trying to dismiss the value there is in having highly scalable cloud architectures available to the projects that need them, but a huge proportion of projects will never need them.

Most projects don't even actually suffer for modest downtime, although you can achieve AWS-comparable downtime even without AWS/Google/Azure-style "cloud" architecture.

It can make a huge difference in operating and engineering costs to know which bucket your project fits in. Good architectural foresight can even let you smoothly move from one to other if unexpected growth or a pivot indicate that it's warranted.

Engineering is about understanding the scope/scale of problems, not about being dogmatic or getting caught up in problems that don't apply.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#96

Post by 300bps »

nojito wrote: post #86
Mo3 wrote: post #80

Earlier quoted context omitted.

You're going to understand where the cool kids come from once your single dedicated server goes down or can't handle the load any more. As soon as you try to scale horizontally or become highly available and start to think about how to do it you end up falling into the same rabbit hole. > If you also use containers you even get quite a bit of flexibility and agility. Yeah... and then the only difference is between a…

Most people don't need to scale and even running 5x redundant servers is cheaper than a comparable cloud solution.

Running Lambda, I get a million calls per month for free. Then it's 20 cents per million calls.

Just curious - have you really researched cloud solutions or did you just compare the price of hosting EC2 instances in AWS vs having your own server? Because that's not what cloud is about.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#97

Post by devmor »

jenny91 wrote: post #30
causi wrote: post #25

Earlier quoted context omitted.

The difference between his situation and yours is that you didn't create the charges. Legally you're not liable for something someone does while impersonating you, even if you walked around with your private key on a t-shirt. They may or may not be nice to him but for you they didn't have a choice.

I don't think that's true? I mean sure, you might not legally be liable when someone impersonates you in the real world. But I'm absolutely certain the AWS terms say somewhere that you agree to take care of your creds and are liable for whatever is done with them, etc?

It doesn't matter what the terms say. The charges would be the result of a violation of Title 18 Code 1030 - it's the digital equivalent of someone stealing your car and writing the title over to someone else. You're entitled to keep your car (or your money spent on AWS) regardless of the receiving party's expectation of claim to it, even if they incurred loss in the process.

Now, Amazon would be entirely within their rights to cancel your account and refuse to do business with you after this, but they would not have the right to collect that money from you, or to keep that money had it already been charged to you.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#98

Post by nojito »

300bps wrote: post #96
nojito wrote: post #86

Earlier quoted context omitted.

Most people don't need to scale and even running 5x redundant servers is cheaper than a comparable cloud solution.

Running Lambda, I get a million calls per month for free. Then it's 20 cents per million calls. Just curious - have you really researched cloud solutions or did you just compare the price of hosting EC2 instances in AWS vs having your own server? Because that's not what cloud is about.

And? I don't think you are aware of how cheap metal is these days.

Plus the OP has already decided that free tiers don't work.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#99

Post by jackson1442 »

huksley wrote: post #84
warent wrote: post #2

Try filing a support ticket with them with this information. I had something vaguely similar happen on GCP and they refunded the full amount

Hi, author here! Thanks, already done that, and waiting for it to be reviewed by AWS. Support was very responsive, though.

Make sure to set up budgets too! Support will gladly help you with that since it helps prevent this situation in the future :)

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#100

Post by pixl97 »

bombcar wrote: post #36
hnlmorg wrote: post #24

Earlier quoted context omitted.

I've experienced plenty of scenarios where costs have quite legitimately spiked. Ultimately whatever solution you put in place, someone is going to complain about it. At least with the system they currently have in place they can reimburse customers. Whereas it is a lot harder to fix their reputation after they've automatically stopped production services.

Given how easily they reimburse customers, I suspect it's intentional - one can be "fixed after the fact" and the other can't - if your site goes down during a slashdotting and you lose sales, etc, there's no getting those back, but if you inadvertently run costs high, they can just refund/cancel those costs.

They 'might' reverse those fees, they might not. You are at their mercy and mercy is finicky.
Post reply on HN