Live data from Hacker News

I got pwned by my cloud costs

troyhunt.com

161–170 of 657 posts

Re: I got pwned by my cloud costs

#161
post #151

Earlier quoted context omitted.

I think the case is for big companies that have a hard time attracting IT talent. Like, not in an even remotely IT-related industry, and their headquarters is in a city with no significant tech community. Places that scream "working here will not improve your resume". I'm a major cloud skeptic, but there's a certain class of giant enterprisey companies that are never going to be able to attract good IT talent, and if…

I believe renting dedicated servers is often overlooked. You pay someone else to install hardware, ensure network connections and be on-site for hardware swaps etc but still have the maximum degree of flexibility. Even larger companies can work well with that model, traffic also tends to be cheap enough that you can spread across different vendors to avoid lock-in. And in that case, your sysadmins can sit wherever th…

I don't have first hand knowledge but my understanding is large companies have procurement departments and they get over half off sticker price for Azure. My guess is this is why the sticker price needs to be overinflated because people in procurement need to show that they are doing their jobs.

Also it is a major pain point getting anything done with IT operations.

Like the Oracle database server that half the department relies on stops responding on a Friday morning and it takes all day to determine the hard disk is full and fix it. I had never before worked at a company where this happened multiple times.

Or operations saying they were unable to restore a windows server hosting a database server and now everyone has to scramble to update their connection strings because operations somehow cannot use the same domain name for the new machine.

Re: I got pwned by my cloud costs

#162

Earlier quoted context omitted.

€40 gives you a dedicated server, not just a VPS. Getting metrics on that is not a hard problem, there are various projects that are relatively simple to set up. If you want to make it easier manage resources, metrics out of the box, and avoid (hoster) lock-in then I'd use a hyper visor distro like Proxmox VE (disclaimer, am a dev there) or the like, and you can migrate (or backup/restore) VMs or Containers easily to…

To make a fair comparison you need to consider the time cost for setting all of that other stuff up compared to having it out of the box. I'd an engineer on 100k takes a week to get it up and running then your vps cost 2k to set up and 40/month going forwards.

If an engineer needs that much time you have a serious technical debt for setup of your software or a inexperienced or inept engineer. The developer or at least operators need to be able to setup your software for more frequently for testing anyway, if they cannot do that rather quick you got other problems..

I can set up Proxmox VE as hyper visor, some container for each DBs, load balancer in front and some app in about an hour max from scratch, with good testing and some bells and whistles, and here I really do not want to brag or the like, as such operations are not my job to do at all, I only know because I do that occasionally for some tests and for some private infra I just maintain out of interest - so I really want to say, if some operation-dork can do that, the engineer you hired should be able to do it at least as quick.

But yes you're right in the general point, upfront setup and frequent maintenance is naturally something you need to price in. I just think that if you have that many different parts with complex coupling to induce such a huge maintenance effort required to keep your product running, the cloud offer may not really be your salvation and just delay the fall while costing all the more.

Re: I got pwned by my cloud costs

#163
post #154

Earlier quoted context omitted.

Being able to run a relatively simple global cache with a cheap provider like Hetzner has the origin is also harder than it should have to be.

How's that? Setting up a revers nginx proxy with cache takes probably less than an hour even if you've never done it (speaking from experience). And otherwise, if the files don't change that much just ssh in, copy them on the server and serve them via nginx and cloudflare tunnel? I'm in no way a sysadmin and have set up these configurations manually in less than an hour for side projects. Cloudflare tunnel also allow…

The usual answer you'll get is that it's not "infrastructure as code", is not highly available, etc... and while that's theoretically true, in practice modern hardware is reliable enough that I'll take the gamble (and the complexity of clouds and their control plane means that you may have more outages than what would be caused by hardware failures).

Re: I got pwned by my cloud costs

#164
post #21

Earlier quoted context omitted.

He's a "Microsoft Regional Director and MVP" so Microsoft pays the bill one way or another. I expect that he has reduced Azure rates as well.

Regional Directors and MVPs aren't employed by Microsoft: https://rd.microsoft.com/en-us/about/

I have a monthly azure credit of $150 and some reduced pricing simply by having a ms developer subscription. I'm guessing Microsoft MVP's (in general, and Azure MVPs perhaps in particular) have extremely generous azure credits so hopefully he isn't on the line for the full amount here.

Re: I got pwned by my cloud costs

#165

Earlier quoted context omitted.

Because it's not cool, and won't make your CV sparkle. I'm sure there becomes a point where cost of (hardware + maintenance + staffing) > (cloud + staffing), in which case sure crack on. But like you, I'll stick to a rented server for my stuff.

The direction is opposite IMO. As you grow bigger on prem starts making a lot more sense.

Not really, on site makes sense for Facebook or Google. Or for extra privacy.

Mid-sized companies can get cracking deals (like 10% cost) on major cloud providers.

Re: I got pwned by my cloud costs

#166

Earlier quoted context omitted.

> I just don't understand why people use cloud services. 1, when they need to adjust rapidly between different resource usage profiles, e.g. because they are growing rapidly and can't predict what the usage will be X days in advance 2. They have huge resource requirements and don't care to invest in their own infrastructure, but can negotiate lower rates with a cloud provider 3. When their resource usage is modest bu…

> 1, when they need to adjust rapidly between different resource usage profiles, e.g. because they are growing rapidly and can't predict what the usage will be X days in advance One can add new servers in minutes, removing has a bit more latency to it, but I'd figure with the huge price difference between rented and cloud you'll come out on top with the former in most case. Also, just use a clustering or orchestratio…

> One can add new servers in minutes

With Vercel I don't ever think about adding servers at all, huge win.

> infra costs are relatively low compared to salary costs

Enterprise SaaS here, this is it. Any second my team spends not caring about infra is well worth it.

Re: I got pwned by my cloud costs

#167
post #137
post #84

Earlier quoted context omitted.

That dedicated server you have to manage (ensure security, install the software you need, keep it updated and secure etc). It’s not for everyone. Also, as you can see in a screenshot on TFA: Some services are simply dirt cheap. The storage account and its various “sub-services” is such a thing. It’s hard to compete with dedicated hardware here. Depending on your dedicated hosting provider, the traffic cost trap exist…

> That dedicated server you have to manage (ensure security, install the software you need, keep it updated and secure etc). It’s not for everyone. apt install unattended-upgrades. And Hetzner's firewall.

By the way, unattended-upgrades is enabled by default nowadays.

Re: I got pwned by my cloud costs

#168
post #36

I wonder if before cloud computing, has there ever been a successful product / service where it was accepted with just a shrug that the volatility of monthly costs means it could bankrupt you with next month's bill, because of complexities and opaqueness of the cost structure make it virtually impossible to predict and protect against extreme peaks in all parts of the setup. Even if you run a relatively opaque cost s…

> Even if you run a relatively opaque cost structure business like a restaurant, you can still calculate the maximum cost of ingredients for one month, the salaries, energy, etc.

If the restaurant suddenly ordered ten thousand times more ingredients than usual, their supplier would probably call back and say "is that really what you want?" rather than just shrugging and shipping them tonnes of tomatoes with a bill for one billion dollars.

Re: I got pwned by my cloud costs

#169
post #105

Earlier quoted context omitted.

Perhaps for the same reason that the vast majority of the readers of this site don’t use Hetzner: they are not European and neither are their users.

The reason Europeans tend to favor European service providers generally has to do with strong data protection guarantees and some level of protection against foreign surveillance. In practice a lot of European companies still use US services or at least services provided by US companies -- Troy Hunt is Australian and uses Azure from Microsoft, so this isn't just a thing Europeans do either. I'd love to hear your reas…

When deploying pet projects I could not care less about privacy.

Re: I got pwned by my cloud costs

#170

Ouch. If Troy Hunt of all people can make this mistake, it can happen to anybody. HIBP is an awesome service funded totally by donations, so it's too bad this happened. Of course Microsoft is happy to hide behind their confusing pricing model and let customers overpay for Azure without alerting them.

Putting anything internet-facing on the cloud is as irresponsible as posting your credit card number publicly. Anyone can essentially charge you an infinite bill and you can't do anything about it until it's too late.

Maybe it's not a problem when you're dealing with millions of VC money, but there's no way in hell I would host anything in a bandwidth-metered cloud service when my or my own company's money is involved.

Post reply on HN