Live data from Hacker News

Tell HN: Google Cloud suspended our production projects at 1am on Saturday

news.ycombinator.com

341–350 of 531 posts

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#341

This is why my old hosting company changed its auto disconnection code to work at 11am on a Monday morning, never at a weekend. That way it's the bill-payers that notice and get in touch, not the poor tech on the out-of-hours rota. Also, it only firewalled the customer's public IPs from the Internet - it didn't stop servers. That way when the account is settled there was often nothing for the techs to do. Also it was…

Which hosting company are you referring to?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#342
I was really under impression that Thomas Currian will make Google cloud a little more reliable. Less cool (even worse tech ) but more reliable: good documentation, reliable servers, good management console, and good support (think Rackspace in olden days).

I guess that is not happening. AWS is good enough and Azure is a mess but at least support personal is trained.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#343
post #264

Earlier quoted context omitted.

For Google a customer is a data point in their machine learning set (hopefully weighted by revenue). The decisions are made by a ML model. If you treat your relationship with Google as such from the beginning, you can extract some value if you use Google for near zero value use cases - spam/marketing emails, disposable non-real time computations, Adwords/AdSense on spammy sites. Not sure if having an Android phone is…

This, either be big enough for a dedicated account representative (do they even offer that?) or use for noncritical workloads.

Does this not cover everything people need right here?

$29/m + 3% of monthly spend https://cloud.google.com/support/docs/standard

Or alternatively $500/m + 3% of monthly spend https://cloud.google.com/support/docs/enhanced

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#344

The same is true of DigitalOcean, fwiw. Shutting down machines used in production with no notice, outside working hours. I don’t know why these companies don’t care - they’re massively shooting themselves in their feet. Incidents like this turn customers off for life.

they’re massively shooting themselves in their feet

It doesn't matter. Google operates at scale. As long as they save more money by not bothering to handle such incidents better, they will see it as a perfectly good business policy.

Incidents like this turn customers off for life.

Small customers are completely interchangeable for larger service providers. You can easily get new customers by throwing around some free credits to college kids and posting a few tech ads masqueraded as tutorials.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#346
post #6

I wonder what it would take to bring a culture change to Google so they can finally start having some empathy for customers and stop the casual cruelty. A new CEO?

It won’t happen. Google is probably going to be like the next IBM: a former tech giant that became irrelevant.

I wouldn't be so sure that will happen to Alphabet before it does to Amazon or MS. Nothing lasts forever, though IBM aren't actually showing any signs of dying outright yet, despite the fact their own attempt to get into the cloud platform market hasn't exactly been inspiring (even if they were only ever intending to cater for niche markets).

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#347
So as of today, mid 2022 ... there is still again ONE only reliable cloud?

How can a company fight the so called "vendor lock-in" without opening their own data center?

The only alternative I see is just really going against the flow - use simpler hosting services, etc.... but who does that nowadays?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#348

Earlier quoted context omitted.

All - absolutely all - the evidence we have seems to show that Google was completely wrong - that a payment didn't go off because of a known Google outage, that the company then made a manual payment, but started to receive threatening messages, that they appealed to billing who told them they were safe, but then the site was cut off in the middle of the night on the weekend. Your argument appears to be this: "Facts…

Epistimological check-in: we don't actually have any evidence, except for a very high level story from the OP without much relevant detail. So all we have is the base rate, which is in almost all cases when there are billing problems they are at least to some extent because of negligence or fraud on the part of the customer.

The base rate of building problems is that the customer screwed up, and the base rate of Google support problems is that Google is negligent. I'm not sure how to compare these.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#349
post #16
post #6

I wonder what it would take to bring a culture change to Google so they can finally start having some empathy for customers and stop the casual cruelty. A new CEO?

Regulation, full stop. The reason Big tech makes so much profits is because they have no accountability and at their scale, social shaming doesn’t work.

[deleted]

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#350
post #315

Earlier quoted context omitted.

How are you supposed to trust a company who told you they wouldn't screw you because of their own error, then screwed you anyway? Having a relationship with them doesn't matter if they still can't be trusted to do business. The type of payment method you use is not a factor that plays into whether you can trust your vendor.

I don't think you can. My takeaway from this discussion is definitely that GCP is only suitable for toy projects, and not for serious production stuff that requires continuity.

This is a fundamentally unserious point unless you want to add more qualifiers.

I’m pointing to this as evidence https://cloud.google.com/customers

I feel like a lot of the problems that people run into with GCP are avoided with two main components.

Number one you should look seriously at the “enterprise” designation for different products and services if you are looking for long term stability and guarantees about the future (details: https://cloud.google.com/apis/docs/resources/enterprise-apis) and secondly you need to buy actual customer support as an add on if you want to avoid the “I had to hit the front page of HN to actually speak to someone” scenario https://cloud.google.com/support

Post reply on HN