Live data from Hacker News

Tell HN: Google Cloud suspended our production projects at 1am on Saturday

news.ycombinator.com

371–380 of 531 posts

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#371
post #360

Earlier quoted context omitted.

Fly.io employee here. Account suspensions are manual. If you have billing issues at most you will be limited on what you can do (deploy, scale...).

exactly. I heard only the good words about fly actually :) You don't break things like google services with their clients.

for my case with google ads I don't even get any reasons to be suspended and no luck to fix it somehow. I was promoting my indie games with no income actually.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#373

Earlier quoted context omitted.

Google tech is the best. Google invented k8s, their k8s cloud is the best in the world. There's nothing similar to GKE Autopilot. Similar things could be said about some other services. People want the best cloud and they hope that those issues won't affect them.

Google has some good tech and then most part of the cloud is just years behind. Want an example: you can't configure the gcp docker registry [EDIT: artifact registry] to cleanup images automatically (after a number of days or number of images or anything). This is total standard in AWS and the ticket for this on GCP has been open for many years now.

Isn’t GCR backed by a GCS bucket in your own project? Can’t you configure object lifecycle management on said GCS bucket?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#374
post #15
post #9

Earlier quoted context omitted.

The only entities Google doesn't treat like garbage are advertisers. All others are part of the product

Haha, I am an advertiser on Google and trust me, getting hold of someone is like doing a summoning ritual.

At least most rituals are well documented ;)

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#375
post #124

A few years ago I was working at a startup and we had just started moving some stuff over from AWS to GCP. Things were going pretty good with GCP until one day they pulled the plug on everything with no explanation. It turns out that our company credit card details had been fraudulently used without our knowledge. The criminal had decided to use the card to buy Google Ad Words or something like that - this is the sam…

Credit cards in general sound like a shitshow that maybe the consumer have the stomach for, but it should never be used for critical business processes. Doesn't Google accept more reliable forms of payment?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#376

Generally, all cloud providers have two ways of engaging with and billing customers. One is using a credit card, which is very easy for getting started. The other one is having an established relationship with a sales / account team and setting up billing by invoice. Smaller companies sometimes don't understand that and stick with credit card billing for too long. Once you move to production and have your own custome…

How would invoice billing even reduce the risk of this outcome? Surely any system with more manual and asynchronous steps involved is MORE likely to result in Google erroneously claiming you haven't paid, not less? I don't grasp what the logic behind this suggestion is even meant to be.

Invoices are indefinitely postpaid; there’s no real point at which you can really 100% declare an invoice as having “not been paid.”

That’s why NET30/90/etc exist — without an incentive to pay sooner, invoices will just sit around on the receiver’s books until it’s convenient for them to pay (e.g. to make their financials for a particular quarter look good.)

Invoices under contract law are like PayPal’s arbitration process: “always favours the buyer.” A seller would need an explicit pre-existing contract clause stating a due date for an invoice, in order for a due date declared on an invoice to have any force; by default, it doesn’t. And because invoice billing is something companies tend to offer mostly to companies they’re trying to stay on the good side of, they don’t tend to sit them down for a binding contract negotiation when doing so.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#377
post #142

Generally, all cloud providers have two ways of engaging with and billing customers. One is using a credit card, which is very easy for getting started. The other one is having an established relationship with a sales / account team and setting up billing by invoice. Smaller companies sometimes don't understand that and stick with credit card billing for too long. Once you move to production and have your own custome…

What is this fud? Both AWS and Azure won't suspend your account for several months, even with charges failing. Especially not if you're communicating with them. It's incredibly easy to get a hold of human support in case of the other clouds.

Had AWS payment issues at one company spending ~$20k/mo. Had some technical issues trying to catch the bill up. Contacted support, obviously concerned. It was entirely a non-event to them. Basically just said as long as you're in contact with us, we're not gonna shut anything off don't worry about it. And... they didn't.

At other companies where we've had an active and involved account rep literally just shot off one email and got back a "no worries, I put a block on the account so it can't be suspended without my approval".

I know in the past I've had personal accounts go a couple months in the red when a card expired and I wasn't staying on top of the emails.

These sorts of things are really only an issue with one specific provider.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#378
post #264
post #124

A few years ago I was working at a startup and we had just started moving some stuff over from AWS to GCP. Things were going pretty good with GCP until one day they pulled the plug on everything with no explanation. It turns out that our company credit card details had been fraudulently used without our knowledge. The criminal had decided to use the card to buy Google Ad Words or something like that - this is the sam…

For Google a customer is a data point in their machine learning set (hopefully weighted by revenue). The decisions are made by a ML model. If you treat your relationship with Google as such from the beginning, you can extract some value if you use Google for near zero value use cases - spam/marketing emails, disposable non-real time computations, Adwords/AdSense on spammy sites. Not sure if having an Android phone is…

GrapheneOS.org, use sandboxed webapps &/or profiles.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#379

Earlier quoted context omitted.

They are, in Europe, where we have decent regulation and consumer protection with teeth.

I'll buy that when the German auto makers get to pay properly for Dieselgate. Until then, EU regulation will be just a demonstration that money and power still talks just like in the US, the only difference is that Europeans are better at fooling themselves about their moral superiority.

This is a non-sequitur. EU has shown plenty of effectiveness enforcing consumer-facing regulation especially in tech/internet.

Dieselgate is a case of corruption in Germany, a particularly horrific and embarrassing one sure. But the existence of corruption in a single member state is not sufficient to disregard EU-wide regulatory efforts in a broad swath of consumer-facing industries.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#380
post #294

Googler here. I'd like to help. DM me https://twitter.com/david_klanac

Are you someone with authority over Billing, or a nice but powerless employee in a different department?

It doesn't take authority to effect change. You'd be surprised at what you can accomplish once you figure out how who owns the various fragments of the solution.

OP – My offer to help still stands. I'm sorry to hear about your bad experience. Sorry doesn't fix it. I realize that. But I want to make it right.

Post reply on HN