Live data from Hacker News

I got pwned by my cloud costs

troyhunt.com

461–470 of 657 posts

Re: I got pwned by my cloud costs

#461

Earlier quoted context omitted.

Except in what minute of my day am I supposed to take off the hat I'm currently wearing to put on my IT Server Room hat? I don't have time to wrangle this stuff any more. I have multiple clients, I have side hustles, I have what's left of a social life after pandemic, I have family obligations. There are only so many hours in a day. If my time become more effecient by throwing a bit of money at the problem, then it i…

There is a difference between freelancers/one person companies and big cooperations though. Honestly, the question you need to ask in regards to cloud is a relatively simple one: Can I hire a sysadmin for cheaper than using the cloud? The answer to that, once you start using enough resources, is more often than not yes. Sure, it takes a while to get to that point, but eventually you will reach break even and it would…

Yes, but these threads of "in house is cheaper than cloud" never qualify at what size company, at what revenue being generated, etc before their version of an answer is true.

I have been on both sides. Large media production companies with very large amounts of fast and redundant storage located on-prem. These range from local attached RAIDs to large shared SAN pools. Their clients also tend to be the types that sue the crap out of you if any of their content is seen by people outside their control. Switching to cloud solutions was (still is) a huge uphill battle. However, the cloud storage needs are no where near the same (not editing content from s3), but storing approved masters for distribution totally makes sense for cloud. Now that the content is in the cloud, why not perform actions on that content in the cloud. Faster deployment, better equipment, blah blah. Next thing you know your entire workflow past editorial is in the cloud. You start to analyze your expenses and compare them to on-prem amortized expenses and you see that it could be cheaper on-prem. Also, take into consideration how long it takes to bring up that new data center. You also have to look at bandwidth expenses. Bandwidth to a new site not directly on the backbone tends to be expensive for non-residential connections. The additional power expenses of that new equipment plus the cooling is also a new expense. Power redundancy you ask? $$$ Now, you need that sysadmin and possibly a small team. At that point, you go back to your cloud rep, and renegotiate fees. You have now created an entirely new department at your company on managing the on-prem.

Re: I got pwned by my cloud costs

#462
post #68
post #6

Earlier quoted context omitted.

Every cloud makes this mistake easy! You have to manually activate billing alerts for everyone because they want you to spend more snd more each month. I am still waiting for a cloud without these dark patterns. But that will never happen because it‘s leaving a big amount of money on the table by not being hostile.

Dark patterns - this sounds like a colour scheme you don't care for. "Predatory death-trap pricing" captures the spirit of the thing with rather more clarity. It is wholly intentional after all.

We had a similar situation to Troy's where several thousand pounds was charged in a matter of days as a result of our misconfiguration of caching in our azure app services (before that month we typically had around £800 a month costs). We emailed Azure / Microsoft and they were happy to refund us. I don't think this is their intended business model.

Re: I got pwned by my cloud costs

#463

Earlier quoted context omitted.

> People think baremetal is hard. It isn't. It does take knowledge. This. I always wonder how much of the "clouds" success (economic, that is) would have materialized, if the marketing term never got traction, and everyone just called it what it really is: "renting someone elses hardware without physical access, and less, if any, control over how the stack works from the metal up". In the good 'ol days, when people w…

No, no. The costs cloud saves are in staffing and opportunity costs. Everyone knows that it is more expensive than a comparable server, but...it is easy, standard, and available. If you want to a) not have real estate capex, b) not worry about the core ops part of your applications, and c) used to outsource infrastructure to a managed service provider anyway, then Cloud is a viable value prop. Plus, the more of the s…

> but...it is easy, standard, and available.

So is a LAMP stack on a dedicated machine.

> Plus, the more of the services you use,

The thing is, most webapps don't use a lot of services. Backend-Logic in whateverlanguage, a database, and a webserver. Maybe hooked up to some CRM system. That's it for 99/100 webservices.

Yes, the services cloud providers offer are amazing, they are complex, and it is natural for developers to be fascinated by complex things (I know it is for me). But it's important to realize when simple is simply enough.

Re: I got pwned by my cloud costs

#464

Earlier quoted context omitted.

Except in what minute of my day am I supposed to take off the hat I'm currently wearing to put on my IT Server Room hat? I don't have time to wrangle this stuff any more. I have multiple clients, I have side hustles, I have what's left of a social life after pandemic, I have family obligations. There are only so many hours in a day. If my time become more effecient by throwing a bit of money at the problem, then it i…

There is a difference between freelancers/one person companies and big cooperations though. Honestly, the question you need to ask in regards to cloud is a relatively simple one: Can I hire a sysadmin for cheaper than using the cloud? The answer to that, once you start using enough resources, is more often than not yes. Sure, it takes a while to get to that point, but eventually you will reach break even and it would…

even worse, i bet a lot are in the situation of "i have cloud AND i need to hire a system administrator".

Re: I got pwned by my cloud costs

#465
post #168

Earlier quoted context omitted.

> Even if you run a relatively opaque cost structure business like a restaurant, you can still calculate the maximum cost of ingredients for one month, the salaries, energy, etc. If the restaurant suddenly ordered ten thousand times more ingredients than usual, their supplier would probably call back and say "is that really what you want?" rather than just shrugging and shipping them tonnes of tomatoes with a bill fo…

I'll bet Sysco would deliver $10k worth of canned tomatoes to your restaurant without checking.

At my previous housing co-op, the new kitchen manager accidentally ordered nine cases of limes (~$1000) instead of 9 limes.

They assumed it was a mistake and only delivered a single case, which was still 180 limes, but at least it didn't use up our entire food budget.

(Normally I'd expect a phone call or email to confirm, but this was a smaller, local supplier, so they probably didn't have real systems to deal with outliers.)

Re: I got pwned by my cloud costs

#466
post #107

Don't put Cloudflare in front of a Cloud egress bill. i.e. don't do this: Azure|Amazon > Cloudflare Always use your own proxy where the egress is well within your free tier, i.e. do this: Azure|Amazon > Hetzner|Linode > Cloudflare Why? Because Cloudflare cache is a massively multi-tenant LRU cache and whilst hot files will be cached well (and with Cloudflare Tiered Cache even better - but this itself is a cost) anyth…

[deleted]

Re: I got pwned by my cloud costs

#467
post #257
post #134

Earlier quoted context omitted.

They'd rather refund small guys for mistakes than give big guys an easy limit to set.

I guess big guys don't want they service to suddenly stop, so they probably would not use this... But it's just a guess

Absolutely that. Storage costs money, so in order to absolutely cap your spending they would have to delete all your stored data, too. Deleting S3 buckets and EBS volumes on a spending blip is absolutely the last thing any company with any budget at all wants to happen, ever. It would be preferable for that not to even be possible in any situation. This is the sort of thing that only extremely small casual users want, and it isn't worth it to AWS to cater to those users. For everyone else, more complexity than a "kill everything at $X" switch is needed, and that's exactly what we do have. We don't get to absolutely cap our spending to the penny but we also don't risk having our data vanish because of a billing issue.

Re: I got pwned by my cloud costs

#468
post #107

Don't put Cloudflare in front of a Cloud egress bill. i.e. don't do this: Azure|Amazon > Cloudflare Always use your own proxy where the egress is well within your free tier, i.e. do this: Azure|Amazon > Hetzner|Linode > Cloudflare Why? Because Cloudflare cache is a massively multi-tenant LRU cache and whilst hot files will be cached well (and with Cloudflare Tiered Cache even better - but this itself is a cost) anyth…

Or simply use a proper CDN that doesn't pretend to eat all the cost for a flat fee but then sometimes does not. BunnyCDN has an amazing volume tier at half a cent per GB.

Will BunnyCDN reliably keep an 18gb file in cache without hitting origin? I use and like Bunny, but relying on that to not get a massive bill in the mail scares the shit out of me.

Re: I got pwned by my cloud costs

#469
post #106

Earlier quoted context omitted.

I don't think Azure offers a Cloudflare alternative, and I'm not sure they ever would - Cloudflare is too good and too cheap to compete against.

It's not that good. I'm constantly getting "Access Blocked" to various websites by Cloudflare trying to protect them....from me reading them.

I don't get the "Access Blocked" often, what I do get is the "We are verifying your browser" page that often just keeps looping there and completely block my access.

Re: I got pwned by my cloud costs

#470
post #329

Earlier quoted context omitted.

Wouldn't riding a horse prevent that car crash?

Luckily you can avoid both by just cycling everywhere. Lower CO2 output and lower cost, too. I use rented dedicated servers for everything, and always travel by bicycle or transit. It's not as ridiculous as you make it seem.

If I ride a bike from work to home, my fat ass will be terrible unhappy. If I ride a horse, the horse would be. I could drive, but driving in London is not fun. Luckily, there is a decent public transport system that fits my needs. The point is, there is a context for everything and it matters.

You might like installing and configuring software, I don't. I'm more than capable of doing so myself, but I'd rather build things on top of other things. I'd rather use a battle tested Secrets Manager and have db replication set up for me. I'm grateful to people that like doing these things I don't and I'm expressing my gratitude by contributing to their paychecks via my cloud bill.

To go back to my initial reply, if you change the context, eg the context is driving a car, you can't possible crash the car you are not driving. If the context is "get home after a few too many pints at the pub", then riding a horse is much better than driving a car (and crashing it). Context.

Post reply on HN