Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Cloud customer wakes up to $18,000 bill despite $7 budget

tomshardware.com

Post reply on HN
Print view View original

41–50 of 69 posts

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#41

Post by Havoc »

GCP is turning into a circus.

Right now there are about 4 concurrent threads on the googlecloud subreddit about people getting hosed with life changing bills. Some no doubt through stupid mistakes (happens), but still bewildering that Google is insisting individuals like students are subject to the same scale to infinity bills as huge corporations and are unwilling to provide any mechanism to protect hobbyists.

And then people tell you but there are quotas and then:

>Google automatically upgraded it to the next level when the account crossed the $1,000 threshold during the incident.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#42

Post by Havoc »

pradn wrote: post #33
victor106 wrote: post #11

Why doesn’t GCP provide a way to say “shut down all my services if my cap is reached”?

Suddenly turning off services when the billing cap is reached is a big reliability risk for customers. All workloads using that billing account would be impacted immediately. If they weren't turned off at the billing cap, but were given some leeway instead, either that becomes the new hard limit, or GCP will have to give away the difference. And there's no "middle ground" you could implement that makes sense either -…

>Suddenly turning off services when the billing cap is reached is a big reliability risk for customers.

That's why you make it opt in with suitable disclaimers about data loss from deletions (a warning which btw GCP already shows you when you disable billing on an active project).

I'd say the average person is well positioned to determine whether they're a broke student that can't afford a $1000 surprise or a corporation that needs reliability above all.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#43

Post by ekimekim »

pradn wrote: post #33
victor106 wrote: post #11

Why doesn’t GCP provide a way to say “shut down all my services if my cap is reached”?

Suddenly turning off services when the billing cap is reached is a big reliability risk for customers. All workloads using that billing account would be impacted immediately. If they weren't turned off at the billing cap, but were given some leeway instead, either that becomes the new hard limit, or GCP will have to give away the difference. And there's no "middle ground" you could implement that makes sense either -…

> Suddenly turning off services when the billing cap is reached is a big reliability risk for customers.

And yet this is exactly what happens if you exceed a quota limit, which aren't always well signposted and sometimes require days to get the limit raised. I've experienced multiple hours-long outages due to hitting GCP quotas we didn't know existed.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#44

Post by hak8or »

Looks like another instance, I am dealing with a very similar issue. I didn't even notice any sort of tiers, as I had 17 grand in costs accumulate at a similar pace.

I (a hobbyist running a small side project for a dollar or two a month in normal usage, so my account is marked as "individual") got hit with a ~$17,000 bill from Google cloud because some combination of key got leaked or my homelab got compromised, and the attacker consumed tens of thousands in gemini usage in only a few hours. It wasn't even the same Google project as for my project, it was another that hasn't seen activity in a year+.

Google refuses to apply any adjustments, their billing specialist even mixed up my account with someone else, refuses to provide further information for why adjustments are being rejected, refuses any escalation, etc. I already filed a complaint with the FTC and NYS attorney General but the rep couldn't care any less.

My gripe is not that the key was potentially leaked or compromised or similar and then I have to pay as a very expensive "you messed up" mistake, it's that they let an api key rack up tens of thousands in maybe 4 hours or so with usage patterns (model selection, generating text vs image, volume of calls, likely different IP and user agent and whatnot). That's just predatory behavior on an account marked as individual/consumer (not a business).

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#46

Post by rcbdev »

estimator7292 wrote: post #27
victor106 wrote: post #11

Why doesn’t GCP provide a way to say “shut down all my services if my cap is reached”?

Same reason that you can't cancel a gym membership by walking up to a person and saying "cancel my membership". Predation, pure and simple. Google doesn't care, no one is holding them responsible for predatory behavior. It's profitable to steal from your customers and there's no downside to doing so.

Where do you live that consumer protection is this bad? Of course you can cancel any gym membership at their physical location, as long as you can also sign up at these physical locations.

What idiot lawmaker would allow a system where you can sign up without a computer, phone or fax, but then need these media to cancel what you just signed up for?

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#48

Post by rcbdev »

0cf8612b2e1e wrote: post #36
robotswantdata wrote: post #29

Do yourself a favour and automate unlinking the billing account. Set it to fire when your budget is hit or whatever your risk tolerance is. Yes nuclear option, but I’ll take an hour down time over a $100k unexpected bill

You still owe a debt, even with no card on file.

If Google really is stupid enough to send collectors for this, they have every right to be sued for obviously deceptive business practices.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#49

Post by danny_codes »

hak8or wrote: post #44

Looks like another instance, I am dealing with a very similar issue. I didn't even notice any sort of tiers, as I had 17 grand in costs accumulate at a similar pace. I (a hobbyist running a small side project for a dollar or two a month in normal usage, so my account is marked as "individual") got hit with a ~$17,000 bill from Google cloud because some combination of key got leaked or my homelab got compromised, and…

Recommend switching to Hezner, they have hard billing limits.

This current cloud paradigm of charging indefinitely is insane for normal consumers and many businesses.

Re: Google Cloud customer wakes up to $18,000 bill despite $7 budget

#50

Post by danny_codes »

abrookewood wrote: post #45

This problem exists to some degree for all cloud services. Is there a solution based on the credit card used? Can you get a CC that has a hard limit guarantee? EDIT: I guess that you'd still be responsible for the charges though.

Hezner is the solution, since you can set hard billing limits there.
Post reply on HN