Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I got pwned by my cloud costs

troyhunt.com

Post reply on HN
Print view View original

541–550 of 657 posts

Re: I got pwned by my cloud costs

#541

Post by GordonS »

TriNetra wrote: post #46
oneepic wrote: post #16

It is worth mentioning that the alert itself costs money. So if you're evaluating the alert every 5 minutes on the past 24h of data it can burn a small but surprising amount of money. From TFA it looks like that would be 10 cents per "time series". Or what I translate it to, is 10 cents every 5 minutes (*I think, but I havent used Azure in some time*). $1.20/hour, $28.80/day, almost $900/month. Not too hard to drop t…

Shameless plug: https://CloudAlarm.in (in beta), sends you real alerts usually faster than azure with multiple reminders. It does this daily unless you tell it to shut up for the month for the given exceed. I call it real alerts because it doesn't wait for consumption threshold to reach the way Azure cost alerts do; as soon as it detects that your current cost * remaining days > the budget amount, it'll send you an a…

But surely CloudAlarm relies on the same data as Azure's alerts do? Azure support told me that data is only updated daily.

Also, Azure has an option to alert you beforehand if it looks like you'll go over; struggling to see how your service is any better.

Re: I got pwned by my cloud costs

#542

Post by brimble »

cyberCleve wrote: post #3

Ouch. If Troy Hunt of all people can make this mistake, it can happen to anybody. HIBP is an awesome service funded totally by donations, so it's too bad this happened. Of course Microsoft is happy to hide behind their confusing pricing model and let customers overpay for Azure without alerting them.

There's an entire surprisingly-large industry built around providing better UI to the major cloud providers, so you can actually tell WTF is going on with billing, access control, networking, et c. They're so hostile that it has to be intentional.

Re: I got pwned by my cloud costs

#543

Post by Arnavion »

sparselogic wrote: post #476
nova22033 wrote: post #267

Earlier quoted context omitted.

our department now spend > £1 million per year to AWS in hosting costs. A 20% reduction in those fees could pay for a few sysadmin(s). You can hire a "few" sysadmins for 200k/year?

A 20% reduction would result in ~£800k/yr.

They're saying that if the AWS costs decreased by 20%, they could use the now freed-up money, ie 200k, to pay sysadmins.

Re: I got pwned by my cloud costs

#544

Post by scapecast »

brimble wrote: post #542
cyberCleve wrote: post #3

Ouch. If Troy Hunt of all people can make this mistake, it can happen to anybody. HIBP is an awesome service funded totally by donations, so it's too bad this happened. Of course Microsoft is happy to hide behind their confusing pricing model and let customers overpay for Azure without alerting them.

There's an entire surprisingly-large industry built around providing better UI to the major cloud providers, so you can actually tell WTF is going on with billing, access control, networking, et c. They're so hostile that it has to be intentional.

The underlying issue is that the cloud console is owned by a single product team, and THEY decide what gets exposed - not the underlying product teams for the individual services. At least that's the case for AWS.

The result is that you get a lowest common denominator type of dashboard. And hence a whole industry of providing just a prettier dashboard on top of AWS / GCP / Azure metrics.

Datadog started with a prettier dashboard for Cloudwatch data.

Cloudability started with a prettier dashboard for the Cost and Usage Report.

And also works the other way around. The individual product teams buy development environments to circumvent the console restrictions.

For example, a few years ago, the Redshift team purchased "DataRow".

Re: I got pwned by my cloud costs

#545

Post by lazide »

Ostrogodsky wrote: post #432
Dave3of5 wrote: post #86

Earlier quoted context omitted.

Also the billing alerts is just that an alert. They should have something in place to put a hard cap on monthly spend. That way his free website would go offline when he's spent > $X. As you say they make it hard deliberately. Edit: Turn out Azure have this: https://docs.microsoft.com/en-us/azure/cost-management-billi...

I dont understand why not all online metered providers are forced by law to do this.

Pretty sure fixing deeply technical business to business transparent-but-potentially-terrible pricing models are pretty far down the priority list on things that will get them re-elected right now (not even counting campaign donations).

Contract dispute cases might clarify it, but probably not in the direction any of us is hoping.

Re: I got pwned by my cloud costs

#546

Post by schemescape »

2ion wrote: post #523

This is why I use fixed price offerings for personal projects. A large bill is probably chump change for someone like Troy, for others it's a year or two of savings. The risk is not worth it.

Would you mind sharing the services you’ve found that have fixed prices? I haven’t had much luck finding services like that (although I’m looking in the < $20/month range).

Re: I got pwned by my cloud costs

#547

Post by razzio »

Hope it is okay and not too much off-topic. I just donated. He deserves it for this service!

Fact is that stuff like this can happen. Consider how many variables are in play to determine the final cost of a cloud service it is very much a double-edged sword. Sometimes you cut yourself unintentionally.

So now we all learn from this, I suggest we help him out.

Re: I got pwned by my cloud costs

#548

Post by alfiedotwtf »

Monotoko wrote: post #535
alfiedotwtf wrote: post #117

Earlier quoted context omitted.

> The only thing that would really help were a hard spending limit that stops all services except storage. Sadly, I haven't found a way to do that with AWS

Kill switches in lambda I believe is possible, running when the alert is triggered

Nice, I'll have a look. Thanks!

Re: I got pwned by my cloud costs

#549

Post by alfiedotwtf »

UnFleshedOne wrote: post #512
alfiedotwtf wrote: post #117

Earlier quoted context omitted.

> The only thing that would really help were a hard spending limit that stops all services except storage. Sadly, I haven't found a way to do that with AWS

I just looked at my AWS account and there seems to be a way to set budget, attach alerts to it and attach actions to alerts. For example there is an action to stop EC2 instances. Not sure if other AWS services have something similar, but at least you can kill your instances if something weird happens. Actions weren't there last time I checked (few years ago).

Thank you, I'll check it out

Re: I got pwned by my cloud costs

#550

Post by Hermitian909 »

Dave3of5 wrote: post #416
capableweb wrote: post #200

Earlier quoted context omitted.

I keep hearing this argument against using your own infrastructure again and again, and I'm not sure how true it is. I've worked with teams on both sides, and everyone is gonna have to deal with figuring out how to run at scale, it's just different ways of achieving that. I've worked with teams that manage their own infrastructure with dedicated servers, and not having to think about scaling for a long time as the on…

Yeah agreed I haven't worked with Google scale companies but I've always found scaling issue to to development related not infrastructure related. So examples would be a bad db query that takes the system down, overly chatting webserver that issues too many queries to the backend, pulling large datasets into the webapp causing exhaustion of memory ...etc. AWS / Azure can't be these issues they have to be fixed in you…

> I've always found scaling issue to to development related not infrastructure related. So examples would be a bad db query that takes the system down, overly chatting webserver that issues too many queries to the backend

This is actually one of the strengths of the cloud, startups that can't afford talent throw compute resources at the problem. Running your own servers isn't hard per se, but it requires a certain breadth of less centrally documented knowledge than the cloud and a willingness to fuss. Developers like that can often command higher prices than most startups pay these days :)

Post reply on HN