Live data from Hacker News

Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

news.ycombinator.com

301–310 of 348 posts

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#301
post #256

I'm very much on the boring technology side of things with respect to hosting. 40€ / month gets you a very powerful dedicated server that can easily handle millions of requests per day and performs incredibly well and can be managed easily. If you also use containers you even get quite a bit of flexibility and agility. To be honest I don't really understand the sentiment that developers can get away with not knowing…

> To be honest I don't really understand the sentiment that developers can get away with not knowing basic sysadmin stuff and at the same time have to spend relevant amounts of time, energy and money to get up to speed with cloud solutions, k8s and so on. Agreed. I hear some saying it's nice to deploy on a lambda because you don't need to know anything about the runtime environment. But it's never quite true. As you…

> But the drawback is that now you spend this effort on learning what is after all a proprietary product of AWS. While AWS is massively popular, the knowledge doesn't translate to anywhere else so it locks you in. If you spent similar time learning the basics of Linux deployment and administration, your knowledge is lower level and more general.

Exactly, I learned Linux in late 90s. There are a few new softwares like nginx instead of Apache but I can bring up a new VPS and set it up using essentially same knowledge from late 90s. With a CDN and caching, you really do have a massively scalable service.

Don't get me wrong, I love Cloud but I will never use it for personal projects. It just feels like renting vs owning. My personal projects will have bugs, and I don't want to be on hook for thousands of dollars.

Learn Linux/Sysadmin once, and you can likely use that knowledge forever.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#302
post #50

Earlier quoted context omitted.

I run 1.2 million uptime checks per week, my total AWS bill was $150/mo before I migrated to permanently running VMs - it's definitely doable without trying too hard.

Yes, moving to VM is definitely doable, but now, being a 1 person dev team, it is challenging in maintenance. My fear is, being on a vacation, and suddenly this VM dies. It might take too much time to bring it back online, and I might be out of good network coverage.

Back in days when our services ran on metal or VM, if we got paged during off-hours and had something else going on, a simple reboot almost always fixed the issue.

We, developers, never liked the reboots though, always wanted to find out root cause, so that we won't be paged again. So, I guess, we moved to the cloud. Now we don't get paged in the middle of night.

But yeah if it was my own company and I was on vacation, I rather take 30 seconds to reboot the server instead of worry about paying thousands of dollars.

As for lack of network coverage, you could do scheduled daily/hourly reboots while on vacation, if it makes sense for your service. Or if your an outage will cause massive disruption to your users, then perhaps hire a part-time sysadmin while on a vacation.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#303

I'm very much on the boring technology side of things with respect to hosting. 40€ / month gets you a very powerful dedicated server that can easily handle millions of requests per day and performs incredibly well and can be managed easily. If you also use containers you even get quite a bit of flexibility and agility. To be honest I don't really understand the sentiment that developers can get away with not knowing…

> To be honest I don't really understand the sentiment that developers can get away with not knowing basic sysadmin stuff

It's not that. I used to own a VPS ISP. I know how the infrastructure stuff works.

It's that I don't care. I want to write code and deploy it to a new machine.

The time savings as a developer is enormous.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#304
post #25

I once committed my private AWS keys to a public github repo. A bot scooped it up nearly instantly and spun up many, many ec2 instances that were (probably) mining bitcoins. I received an automated email from Github telling me that I had committed a private key, but it came in the middle of the night. In the morning, when I learned what had happened, my bill was over $3k. I fixed the issue and emailed AWS asking for…

The difference between his situation and yours is that you didn't create the charges. Legally you're not liable for something someone does while impersonating you, even if you walked around with your private key on a t-shirt. They may or may not be nice to him but for you they didn't have a choice.

I don't think Amazon is going to evaluate this on the legal merits.

In a situation like this, Lambda is almost pure profit. Their actual spend here was negligable.

They are almost certain to waive the fee, because they don't want the perception among developers that AWS is a time bomb.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#305
post #256

Earlier quoted context omitted.

> To be honest I don't really understand the sentiment that developers can get away with not knowing basic sysadmin stuff and at the same time have to spend relevant amounts of time, energy and money to get up to speed with cloud solutions, k8s and so on. Agreed. I hear some saying it's nice to deploy on a lambda because you don't need to know anything about the runtime environment. But it's never quite true. As you…

I couldn't agree more. On top of that, cloud tech can't be "set and forget" because of deprecation of apis and services.

I'm sure we weren't the only ones bitten by that in the last few years, when AWS deprecated node 12 and then .NET core 2.1. Yes there were warning emails etc. and there's good reason to keep up with using the latest tools etc. where possible, but at some point you'd like to think "this component is stable, no need to touch it again".

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#306

Earlier quoted context omitted.

If your lambda runs very infrequently, you'll see more cold starts which will take hundreds of ms, but warm starts are pretty easily in the single-digit milliseconds (at least for a Go lambda). If you're doing a bunch of compute or sync I/O in a loop then your functions will take longer. But less frequent invocations are actually an even better case for Lambda versus a VPS because it suggests less wasted time (yeah,…

Yes, cold starts are definitely a problem. One of the advantages of VPSes is its all hot, ready to go, obviously. You pay for that performance, for that lower latency, and I'm fine with that. My big problem with lambda / serverless is the developer experience is pretty awful. The time between making a change, deploying, and seeing the result of that change is slow. You can work around this (with tools like localstack…

Yeah, it depends on the application. If you're very latency sensitive, then you'll probably want to keep your lambda warm or just pay more for VPS. But the original context was cost, not latency.

> My big problem with lambda / serverless is the developer experience is pretty awful. The time between making a change, deploying, and seeing the result of that change is slow. You can work around this (with tools like localstack), but it's often not close enough to the real environment. You'll still waste tons of time debugging permissions issues when you do a real deploy.

I haven't had much of a problem. Once I figure out the shape of the inbound payload, it's pretty easy to test locally. I can't think of any reason the runtime environment would be an issue. Debugging IAM is tedious, but I just created one Terraform module that I use for all of my functions so I don't have to slog through the IAM stuff every time I make a function.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#307

I think every developer has an AWS billing horror story. My horror story is that my site allows users to upload videos and share them to a limited number of colleagues. When a user requests a video, a CloudFront URL is created that lasts a few hours. I had not thought much about hotlinking because the link only lasts a few hours - what would be the point? Well, those few hours make a big difference when it’s linked o…

How can two few gb files = 10k usd of data in a few hours? How popular is that vietnamese porn site!!

Worth noting that if your distribution is set to use every region, Asia-Pacific CF pricing is actually more expensive than raw S3. $0.12 vs 0.09 (for S3) or 0.085 (for North America CF). It's easy to accidentally increase your costs by 33%, since you'd only encounter this with S3 if you put the bucket in an asian region, versus CF where distributions are more hand-wavey about locations.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#308
post #97

Earlier quoted context omitted.

It doesn't matter what the terms say. The charges would be the result of a violation of Title 18 Code 1030 - it's the digital equivalent of someone stealing your car and writing the title over to someone else. You're entitled to keep your car (or your money spent on AWS) regardless of the receiving party's expectation of claim to it, even if they incurred loss in the process. Now, Amazon would be entirely within thei…

Title 18 Code 1030 says it is illegal to commit computer fraud but it is not a responsibility of your service provider to eat/pay for fraud committed against you. Your only legal recourse under Title 18 Code 1030 is against the "violator". Amazon did not violate your computer systems and commit these offenses. > Any person who suffers damage or loss by reason of a violation of this section may maintain a civil action…

You have a fundamental misunderstanding of the positions of the parties in this scenario.

The computer fraud in this case was not committed against you. It was committed against Amazon. Amazon grants you access to their services, the account does not belong to you. The damages here are not made against you, they are made against Amazon.

Just like in my example, the violator committed fraud against the "buyer" of the car. Neither Amazon or the "buyer" have recourse against you for the supposed owed property/bill, they have to extract damages from the violator. You are not responsible.

On your second point, I will repeat myself: it doesn't matter what the terms or contract say. Such agreements commonly hold terms that are in direct opposition to US law and have no legal basis. Their entire purpose is to dissuade you from pursuing your legal rights at a cost to the company.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#310

I'm very much on the boring technology side of things with respect to hosting. 40€ / month gets you a very powerful dedicated server that can easily handle millions of requests per day and performs incredibly well and can be managed easily. If you also use containers you even get quite a bit of flexibility and agility. To be honest I don't really understand the sentiment that developers can get away with not knowing…

> easily handle millions of requests per day

Yes. People don't realize how little that is. Million requests per day is 12 requests per second, on average. Even if we factor in peak load at 20x that, it is still well within reach of exceptionally modest hardware, assuming at least vaguely sane software stack.

Post reply on HN