Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I got pwned by my cloud costs

troyhunt.com

Post reply on HN
Print view View original

591–600 of 657 posts

Re: I got pwned by my cloud costs

#591

Post by goodpoint »

Kneecaps07 wrote: post #352
Dave3of5 wrote: post #77

Ah the old cloud provider switcheroo. Yip this is the way they make money. They make it easy to setup some gigantic hugely scalable website then hit you with a gigantic scaled up bill. AWS would do this as well. Team I'm in at the moment is in the early stages of cloud adoption but the company in total has fell hook line and sinker for AWS. When I mentioned the cost there is always an excuse. The main one being that…

There's an argument to be made for quality of life for your employees. As someone who has transitioned from on-prem server management to mainly cloud work, my job happiness has skyrocketed. I haven't set foot in a data center in three years and I do not miss it one bit. Dealing with hardware failures, hardware vendors, confusing licensing, having to know SKUs, racking new cabinets, swapping hard drives, patching serv…

> Dealing with hardware failures ... it's all awful work

I've met plenty of datacenter technicians that loved they work and the opportunities for growth it provided.

Some companies really know how to manage a datacenter with minimum pain. Some don't.

Re: I got pwned by my cloud costs

#592

Post by lewisl9029 »

canucker2016 wrote: post #580
buro9 wrote: post #107

Don't put Cloudflare in front of a Cloud egress bill. i.e. don't do this: Azure|Amazon > Cloudflare Always use your own proxy where the egress is well within your free tier, i.e. do this: Azure|Amazon > Hetzner|Linode > Cloudflare Why? Because Cloudflare cache is a massively multi-tenant LRU cache and whilst hot files will be cached well (and with Cloudflare Tiered Cache even better - but this itself is a cost) anyth…

Or Troy Hunt can ping his Cloudflare contacts and see if he can get access to Cloudflare R2 Storage. see https://blog.cloudflare.com/introducing-r2-object-storage/ From the Cloudflare blog, it seems R2 would've handled this exact situation - auto-migration of cloud S3-like-storage objects - download from cloud-storage just once and cache in R2 for Cloudflare to serve.

Has anyone gotten access to R2 yet? I signed up but haven't heard back myself.

Would love to find out if you can write to any/every region and have things replicate, or if you have to write to a single region. BunnyCDN's edge storage solution looked interesting until I found out it only supported writes to a single region.

Hoping R2 might be my savior here, otherwise will probably have to roll my own active-active minio cluster, which I'm not looking forward to maintaining. Other suggestions welcome!

Re: I got pwned by my cloud costs

#593

Post by jjav »

dkarl wrote: post #590
jjav wrote: post #554

Earlier quoted context omitted.

> I don't disagree; but I think the cloud (AWS/Azure/GCP) have sort of shielded people from how cheap/powerful the underlying hardware has became. I guess I shouldn't be surprised, but I do find myself often surprised to realize that for a younger generation of developers they have never experienced hosting on bare metal. So they have not been exposed to costs & benefits vs. the cloud approach and feel that no local…

As a software developer, I think the best thing about the cloud is knowing that if you need the capacity, and it makes sense cost-wise, you'll get it. In-house servers might be cheap, but in my experience it could be incredibly hard to get that money spent when it's needed, and I've seen companies throw expensive software engineering time at optimizing software when it would have been much cheaper to solve the proble…

> As a software developer, I think the best thing about the cloud is knowing that if you need the capacity, and it makes sense cost-wise, you'll get it.

Yes, that's the beauty of it and sometimes you need it.

OTOH, how often do you need to grow capacity without any lead time like that? If you are in a hyper-growth stage in a startup you absolutely need it and it is a lifesaver.

But, most companies never see a hyper-growth stage. Even those which do, it's a relatively short timeframe (you can't grow exponentially very long).

All the rest of the time it's a fairly large premium to pay just in case another hyper-growth period happens. Sometimes it's totally worth it. But good to review the likelyhood and cost tradeoffs every now and then.

Re: I got pwned by my cloud costs

#594

Post by Nextgrid »

dkarl wrote: post #590
jjav wrote: post #554

Earlier quoted context omitted.

> I don't disagree; but I think the cloud (AWS/Azure/GCP) have sort of shielded people from how cheap/powerful the underlying hardware has became. I guess I shouldn't be surprised, but I do find myself often surprised to realize that for a younger generation of developers they have never experienced hosting on bare metal. So they have not been exposed to costs & benefits vs. the cloud approach and feel that no local…

As a software developer, I think the best thing about the cloud is knowing that if you need the capacity, and it makes sense cost-wise, you'll get it. In-house servers might be cheap, but in my experience it could be incredibly hard to get that money spent when it's needed, and I've seen companies throw expensive software engineering time at optimizing software when it would have been much cheaper to solve the proble…

> in my experience it could be incredibly hard to get that money spent when it's needed, and I've seen companies throw expensive software engineering time at optimizing software when it would have been much cheaper to solve the problem with hardware.

That's true with owning your hardware, but what about renting from Hetzner/OVH/etc? You get servers set up in minutes unless you have a very specific request (the only time I've had lead time with these providers is when I had a very custom request, a machine with 300+ TB of storage - yes that is not a typo). Everything else has been delivered pretty much instantly.

But even if let's say you have a very specific use-case such as needing a 300TB server that would typically require lead-time, well in that case the prices are so cheap that you can just keep it around all the time sitting mostly unused and still come out ahead compared to cloud pricing.

Re: I got pwned by my cloud costs

#595

Post by Nextgrid »

godot wrote: post #491

These stories almost always boil down to this fundamental conflict of what you want for a personal project vs a business. (though in this case yes, Troy Hunt's HIBP is larger than a lot of startup businesses) In a business setting, you want your service to stay up, at the cost of spike in costs if accidents or mistakes happen. In a personal project, you want there to be hard limit on cost, and your service to go down…

The problem is that the cloud alone will not allow you to scale infinitely. Slapping a standard DB-based application on the cloud will not magically make it scalable. RDS will still be a bottleneck and can't be scaled online for example (Aurora might change that, but that's a very recent development).

Making your application scalable is a significant effort that may involve different trade-offs. Your typical Prestashop or Magento e-commerce site will still max out the DB and go down, cloud or not, but with the cloud you'll end up with a huge bill in addition to your downtime.

Engineering your application to be scalable is an option that's often not made for cost/time to market reasons which is fine, but in this case the cloud will give you much less scalability than a lot of people believe.

Re: I got pwned by my cloud costs

#596

Post by jimbobimbo »

sascha_sl wrote: post #258
buro9 wrote: post #107

Don't put Cloudflare in front of a Cloud egress bill. i.e. don't do this: Azure|Amazon > Cloudflare Always use your own proxy where the egress is well within your free tier, i.e. do this: Azure|Amazon > Hetzner|Linode > Cloudflare Why? Because Cloudflare cache is a massively multi-tenant LRU cache and whilst hot files will be cached well (and with Cloudflare Tiered Cache even better - but this itself is a cost) anyth…

Or simply use a proper CDN that doesn't pretend to eat all the cost for a flat fee but then sometimes does not. BunnyCDN has an amazing volume tier at half a cent per GB.

Azure has its own CDN. If one wants to do Cloudfare -> CDN -> Azure Storage, then at least let it be Azure CDN in the middle, not another cloud provider in the mix. ¯\_(ツ)_/¯

Re: I got pwned by my cloud costs

#597

Post by Nextgrid »

sgustard wrote: post #555
tpetry wrote: post #6

Earlier quoted context omitted.

Every cloud makes this mistake easy! You have to manually activate billing alerts for everyone because they want you to spend more snd more each month. I am still waiting for a cloud without these dark patterns. But that will never happen because it‘s leaving a big amount of money on the table by not being hostile.

If I leave my water running then go on vacation I'll have a huge water bill too. I don't conclude my water company is intentionally trying to overcharge me. The more reasonable conclusion is: building an alert system that addresses every customer need is hard. Most enterprises (where all the customer focus is) want minimal downtime above other considerations, including cost.

The problem with the cloud is that the entire Internet is in control of your tap and can choose to open it and waste infinite amounts of water with no way to implement a cap.

Re: I got pwned by my cloud costs

#598

Post by Nextgrid »

christophilus wrote: post #297
nix23 wrote: post #15

Earlier quoted context omitted.

>Every cloud makes this mistake easy! Funny enough...Oracle (OCI) makes it better, you can buy oracle"coins" 1to1 with $ and load your account just with what you think you need.

If Oracle cloud is still shenanigan free in 2 decades, I’ll consider it. Until then, Oracle gets $0 of any budget I’m in charge of.

The Oracle cloud UI is extremely slow and terrible to work with. I'd prefer shenanigans every so often than having to deal with that crap on a daily basis.

Re: I got pwned by my cloud costs

#599

Post by j1elo »

sydthrowaway wrote: post #561
j1elo wrote: post #32

Earlier quoted context omitted.

> If Troy Hunt of all people can make this mistake, it can happen to anybody. Exactly this. As a low-level / embedded / non-cloud stuff dev, I've been getting up to speed through all the cloud-ification of the industry, but I'm still scared (not literally ofc) of running most things on my own on any big cloud provider (smaller ones seem more manageable). I'm reading this and seems like being a customer of cloud servi…

How are you teaching yourself?

Due to my work, I'm involved in some WebRTC stuff and have been getting in touch with deployments on AWS, CloudFormation, that kind of stuff.

Re: I got pwned by my cloud costs

#600

Post by Nextgrid »

bstpierre wrote: post #477
suction wrote: post #36

I wonder if before cloud computing, has there ever been a successful product / service where it was accepted with just a shrug that the volatility of monthly costs means it could bankrupt you with next month's bill, because of complexities and opaqueness of the cost structure make it virtually impossible to predict and protect against extreme peaks in all parts of the setup. Even if you run a relatively opaque cost s…

A gas or electric bill works a bit like this... if you have some appliance that fails in a way that suddenly starts consuming much more than usual you can end up with a fairly large bill at the end of the month. Same for old school landlines or cell phones, before flat rate billing became ubiquitous. Though in those cases the billing isn't really complex or opaque, and you _can_ monitor it if you care to check your m…

The big difference in case of gas or electric (or even telephone) is that you and only you are in control. Someone would have to physically break into your house or steal your phone to rack up a huge bill.

In contrast, with the cloud the bill is directly proportional to the amount of inbound requests from the Internet, with no out of the box way to implement a limit (I guess you could install Apache/Nginx and enforce a limit there, but doesn't that kinda defeat the whole point of the cloud?).

Post reply on HN