Live data from Hacker News

Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

news.ycombinator.com

131–140 of 348 posts

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#131
Really wish the major clouds had a “I’m experimenting. Kill it if it goes over 1k usd” mode. Not some crappy alerts - actual halt (and delete if necessary)

I bet that would increase profits too by encouraging people try experiment more

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#132
post #36
post #24

Earlier quoted context omitted.

I've experienced plenty of scenarios where costs have quite legitimately spiked. Ultimately whatever solution you put in place, someone is going to complain about it. At least with the system they currently have in place they can reimburse customers. Whereas it is a lot harder to fix their reputation after they've automatically stopped production services.

Given how easily they reimburse customers, I suspect it's intentional - one can be "fixed after the fact" and the other can't - if your site goes down during a slashdotting and you lose sales, etc, there's no getting those back, but if you inadvertently run costs high, they can just refund/cancel those costs.

That’s literally the point I made :)

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#133
I started off using Lambda as well and made the same sort of mistake. I can't remember exactly how much my bill was, but it was enough that it would have drained all my savings and effectively kill my startup. AWS was kind enough to write off most of it.

We now use Lambda only for simple cron / background tasks, or consuming from Kinesis. We use ECS for everything else. ECS is nice because it's relatively simple compared to K8, but still gives the full benefit of running multiple containers on one box.

I wrote a blog post last year about our migration over to ECS and experimenting with various ways to cut costs: https://blog.bigpicture.io/how-we-cut-our-aws-bill-by-70/

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#134
post #80

Earlier quoted context omitted.

You're going to understand where the cool kids come from once your single dedicated server goes down or can't handle the load any more. As soon as you try to scale horizontally or become highly available and start to think about how to do it you end up falling into the same rabbit hole. > If you also use containers you even get quite a bit of flexibility and agility. Yeah... and then the only difference is between a…

For 99% of projects you're never going to hit the point a single server (or group of servers if you truly need redundancy for some level of uptime) can't handle the load. For the other 1% that end up needing that scale I have a hard time accepting it's actually better to start building for massive scale day 1 instead of day 1000.

If the solution you're building is business-critical then you already have a problem day one: you need high-availability. That means you need at least two servers. As soon as you add that additional server your troubles have begun. Doesn't really much matter whether you add 1 server to your setup or 100.

Your problems compound if you have to consider disaster recovery (DR) and need an offsite location you can failover to. Now you have to contend with data replication and all its concerns.

So even if your application only has a few dozen users things can get quite complex quite fast. AWS makes it simple, practically push-button. Deploying in multiple Availability Zones (AZ) is a no-brainer, and it isn't horribly difficult to deploy in multiple regions, complete with data replication. All while having no servers to configure and maintain. No containers to contend with, no Kubernetes to fight with. To me it's the most stupid-simple solution available today and it's ridiculously cheap! At least so long as you test before you deploy for world exposure!

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#136
post #119

Earlier quoted context omitted.

I work in fintech (banking and payments) and most of our clients aren’t even allowed to host on aws; they have to go for a local provider because aws doesn’t have a hosting hub in their country. Not sure what part you work in but this has never been a problem in the past 20 years with just servers, switches, load balancers etc. I prefer aws over metal for these kind of setups, but for many other cases I definitely do…

I don’t know about that, the hedge fund I work for is US-based and AWS can be fully SEC and FINRA compliant. We also have a few dedicated servers, but mostly only for infrequently accessed data and logging that doesn’t need to be highly available. I really can’t understand why this argument keeps coming up. Different solutions for different usecases. Yet anytime Kubernetes or cloud functions are discussed people come…

Pretty easy, OP sounds exactly like a guy who would have been perfectly fine with a single (Hetzner) server. Then these arguments popup and they are most of the time right. I agree with you, that it always depends on the use. However, hurr durr Hetzner Server seems to be the more reasonable choice here (once again).

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#137

I'm very much on the boring technology side of things with respect to hosting. 40€ / month gets you a very powerful dedicated server that can easily handle millions of requests per day and performs incredibly well and can be managed easily. If you also use containers you even get quite a bit of flexibility and agility. To be honest I don't really understand the sentiment that developers can get away with not knowing…

Servers can also be liability. You need to document, implement and maintaing hardening, have a process for regularly patching os and apps, monitor logs, have backup and disaster recovery procedures, regularly test the procedures, figure how to implement data encryption at rest, implement high-availability and so on. Good platform-as-service can solve many things for you and let you focus on the core thing you are pro…

I ran a website with 37 million users and 3.6Gbps peak bandwidth (JavaScript+thumbnails, no video) from my own two racks of Linux servers thay i have not systematically updated for years. The OSes were beyond the lts support winows. I manually compiled my own updates, but very rarely and only those that i deemed critical. Granted, the site stack was completely custom so the standard automated hacks didn't work. In 15 years there were no accidents.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#138
post #119

Earlier quoted context omitted.

I work in fintech (banking and payments) and most of our clients aren’t even allowed to host on aws; they have to go for a local provider because aws doesn’t have a hosting hub in their country. Not sure what part you work in but this has never been a problem in the past 20 years with just servers, switches, load balancers etc. I prefer aws over metal for these kind of setups, but for many other cases I definitely do…

I don’t know about that, the hedge fund I work for is US-based and AWS can be fully SEC and FINRA compliant. We also have a few dedicated servers, but mostly only for infrequently accessed data and logging that doesn’t need to be highly available. I really can’t understand why this argument keeps coming up. Different solutions for different usecases. Yet anytime Kubernetes or cloud functions are discussed people come…

Because some people here do exactly the same thing with aws/cloud. Not you but many here treat aws like it’s the thing you should use ‘because scaling and failover and omg 1 sec downtime’. And then the stories like OP showing that it is dangerous and my experience that most who do this are overpaying and could do with a hetzner server or, better even, a $5/mo vps, even if they do it right (which they are not, generally from all setups I have seen).

I agree with you though on the case by case.

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#139
post #133

I started off using Lambda as well and made the same sort of mistake. I can't remember exactly how much my bill was, but it was enough that it would have drained all my savings and effectively kill my startup. AWS was kind enough to write off most of it. We now use Lambda only for simple cron / background tasks, or consuming from Kinesis. We use ECS for everything else. ECS is nice because it's relatively simple comp…

On a side note, I don't know if you've already acquired any AWS credits. If not, Product Hunt Founders Club is a decent deal that will give you $5k in AWS credits. Between that and the no Stripe fees for 1 year, it paid for itself in no time.

https://www.producthunt.com/founder-club

Re: Tell HN: I DDoSed myself using CloudFront and Lambda Edge and got a $4.5k bill

#140
post #50

Off-topic maybe, but I'm very curious as it's hard to find hard numbers: > my usual bill is $200/month How many req/second are you serving? What kind of things are happening? It seems like the bills are outrageously expensive when it comes to various cloud services, as I'm personally hosting a service that does between 10-100 req/second on average during a month, and my monthly bill end up being closer to $40/month,…

I run 1.2 million uptime checks per week, my total AWS bill was $150/mo before I migrated to permanently running VMs - it's definitely doable without trying too hard.

Yes, moving to VM is definitely doable,

but now, being a 1 person dev team, it is challenging in maintenance.

My fear is, being on a vacation, and suddenly this VM dies. It might take too much time to bring it back online, and I might be out of good network coverage.

Post reply on HN