Live data from Hacker News

Google Cloud customer wakes up to $18,000 bill despite $7 budget

tomshardware.com

51–60 of 69 posts

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#51

Do yourself a favour and automate unlinking the billing account. Set it to fire when your budget is hit or whatever your risk tolerance is. Yes nuclear option, but I’ll take an hour down time over a $100k unexpected bill

You still owe a debt, even with no card on file.

Dangerous litigation for Google to get into. I assume they’d prefer to settle out of court or comp you

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#52
post #4

Earlier quoted context omitted.

The only way I can read that is 'setting a cap does nothing' but reading that tells me that it only turns on email notifications. Not any better really. It's simply not a cap. It's an alarm.

Agreed. I don't agree with the underlying design decisions either, but you literally set a "Budget Alert" on Google Cloud. It's designed to be an alarm, not a cap. I was just trying to point that out

IMO it’s deceptive to market it as a cap. It’s genuinely confusing.

The service page should read “we can charge you an unlimited amount at any time if you make a technical mistake”

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#53
post #28

Earlier quoted context omitted.

aws, gcp, azure (the ones I work with), they don't provide a off the shelf solution to block after some budget ammount. This is not aceptable.

They charge for a lot of things "by the hour". Things like S3, load balancers, storage. Deleting those when a customer hits a limit will lose customer data or remove things that might be hard to add back. The "I hit my AWS limit and they deleted all my data" headlines will result. and excluding those things makes the limit soft again..

Maybe relying on one company to store all the data your company has is a terrible idea

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#54

I think I read somewhere that calculating and limiting cloud usage costs is a really hard problem. But I feel that if Google were motivated to do it, they can do it. It's hard, not impossible. They just don't care to solve this particular problem.

I mean yes, look at Corey Quinn [1] for example. He has built an entire career out of the fact that cloud billing trips people up. (Generally, tech seems to skate by on creating insanely complicated things, knowing that given enough pain, people will start blogging about their solutions, ie effectively outsourcing the cost and effort of doing something about it.) [1] https://www.lastweekinaws.com/

Tech skates by on monopoly/oligopoly power. This arises because big players are allowed to buy competitors whenever they like. And since they are already monopolies/duopolies, they have unlimited money for such purposes. Killing off WhatsApp was chump change for Facebook.

We essentially don’t have monopoly enforcement in the US anymore

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#55
post #46

Earlier quoted context omitted.

Same reason that you can't cancel a gym membership by walking up to a person and saying "cancel my membership". Predation, pure and simple. Google doesn't care, no one is holding them responsible for predatory behavior. It's profitable to steal from your customers and there's no downside to doing so.

Where do you live that consumer protection is this bad? Of course you can cancel any gym membership at their physical location, as long as you can also sign up at these physical locations. What idiot lawmaker would allow a system where you can sign up without a computer, phone or fax, but then need these media to cancel what you just signed up for?

In the United States this is not true. Bally's gym memberships are examples of this. I had one once and there were only three ways to cancel it. 1. You die, 2. You become disabled in a permanent way that would prevent you from ever using it, or 3. You move far enough away that there is no physical location within 100 miles of you AND they won't be building one within the next year.

This has changed since I had mine as I had the membership around 30 years ago, but they still in general do not allow cancellation at the physical location. You must write to them.

Edit: I just read that they all transitioned to LA Fitness now, and I'm not sure what their policies are but Bally's, while unethical and infamous, were not illegal as far as I know.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#56
post #46

Earlier quoted context omitted.

Same reason that you can't cancel a gym membership by walking up to a person and saying "cancel my membership". Predation, pure and simple. Google doesn't care, no one is holding them responsible for predatory behavior. It's profitable to steal from your customers and there's no downside to doing so.

Where do you live that consumer protection is this bad? Of course you can cancel any gym membership at their physical location, as long as you can also sign up at these physical locations. What idiot lawmaker would allow a system where you can sign up without a computer, phone or fax, but then need these media to cancel what you just signed up for?

It may have changed but there are (or used to be) gym memberships that would only let you cancel if you could prove that you moved and your new primary residence was more than one hour from the nearest branch. (I'm pretty sure there was a law passed to end this practice, but it was widespread)

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#57

Do yourself a favour and automate unlinking the billing account. Set it to fire when your budget is hit or whatever your risk tolerance is. Yes nuclear option, but I’ll take an hour down time over a $100k unexpected bill

You still owe a debt, even with no card on file.

When disconnected all services immediately stop. So no big bills.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#58
post #14
post #10

Earlier quoted context omitted.

Depends on latency. 24 hour delays on an eventually consistent counter used for billing absolutely would cause this problem.

It seems hard to believe that a one-hour delay on such a counter is impossible to achieve, and one hour would reduce the risk from "catastrophic" to "serious problem" in most cases. Also, if implementing a cap is a desired feature that justifies trade-offs to be made, then it is psosible to translate the budget cap (in terms of money) back into service-specific caps that are easier to keep consistent. Such as "autosc…

Eh, suddenly turning off all services in your account because you hit your cap is just as much a DoS type event - just of your services, not your wallet.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#59
post #44

Looks like another instance, I am dealing with a very similar issue. I didn't even notice any sort of tiers, as I had 17 grand in costs accumulate at a similar pace. I (a hobbyist running a small side project for a dollar or two a month in normal usage, so my account is marked as "individual") got hit with a ~$17,000 bill from Google cloud because some combination of key got leaked or my homelab got compromised, and…

Recommend switching to Hezner, they have hard billing limits. This current cloud paradigm of charging indefinitely is insane for normal consumers and many businesses.

Switching to hetzner for... Gemini? What do you mean

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#60

Why doesn’t GCP provide a way to say “shut down all my services if my cap is reached”?

Because there's money in letting people make mistakes on their own dime.

For every story about a mistake in the tens of thousands, I wonder how many there are in the single-thousands, hundreds and even tens where people just suck it up and pay the bill. When you provide such a limited billing support surface, employ a thousand lawyers in-house, and hold as many cards as Google does (losing my G account would ruin my year).

Multinational service providers need better regulation to ensure consumers' rights are protected. In the UK utilities and banks have to absorb (or insure) against some leaks, theft and external fraud. If Google (and others!) were subjected to similar regulation, problems like this would evaporate overnight.

Post reply on HN