Tell HN: Google Cloud suspended our production projects at 1am on Saturday • Hacker News Board

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#21

Post by Operyl » Mon, Aug 22, 2022, 11:02 AM UTC

hericium wrote: post #18

Operyl wrote: post #9

Earlier quoted context omitted.

How weird, we get charged 0.01 all the time, and I laugh thinking that it cost them more to charge us a penny than it would've been to just roll it over.

> I laugh thinking that it cost them more to charge us a penny than it would've been to just roll it over In the long run it's not them covering those costs but you and other customers.

Perhaps, but certainly not from me. For the past 2 years it has been a penny a month bill. My statement was mostly that their processing fees are pennies and they could have just waited a bit.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#22

Post by 23Ro » Mon, Aug 22, 2022, 5:06 PM UTC

Disclaimer: not paid, neither working for google.

Ouch, this is a very bad situation. As a business owner of a SaaS offering myself this sounds like a nightmare (and we are on GCP).

Few months ago (in a different company) I had a very positive experience with GCP. We've unfortunately had a situation where a over-provisioned service account leaked. It has, of course, been used to mine crypto by some malicious actor. We've ramped up a very high six figure bill in just a few hours of this going unnoticed. After reaching out to google, and a months long review process, the bill we've ramped up has been dropped. During the waiting period we have NOT paid any bill for the entire time. Granted: we've been told to not pay by the customer service, until the situation has been sorted out.

What I'm trying to say is: there's good and bad, at every major cloud provider.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#23

Post by Dr_ReD » Tue, Aug 23, 2022, 12:36 AM UTC

While I empathise with the OP, I would add:

- never-ever really trust any company assurances or contracts. Especially if they're way bigger than you are, always assume they WILL fail you and you'll have no recourse outside of litigation—and even then... :-/

Always have a backup plan for important stuff. Always. No exceptions.

More in general, avoid every single point of failure you can afford to.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#24

Post by splix » Tue, Aug 23, 2022, 2:55 PM UTC

Same happened to us the last Thursday, but in our case they suspended our GCP account without a prior notification. Just learned about this from our external monitoring.

Turns out they wanted to check my IDs, and it's after I had been their customer for 14 years. And it took them 5 days to verify that ID. An hour ago they activated it back.

I've been running a few projects on GCP, with active customers, which I lost now. Those 5 days of outrage is basically a catastrophe for us.

And no apology was given by Google. There was barely any communication from their side.

Never use GCP for production.