Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Google Cloud suspended our production projects at 1am on Saturday

news.ycombinator.com

Post reply on HN
Print view View original

441–450 of 531 posts

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#441

Post by carapace »

andrewinardeer wrote: post #88

I know I'm going to come off as a Google apologist, however, I'm leaning towards there being more to the story than what you are saying. The previous month's billing didn't 'went through' and you are unable to zero in on whether this is a Google issue or a you issue. I get that it is easier to complain on the internet and often cathartic do so as a coping mechanism. Having an account suspended because of a billing is…

(I just wanted to say thank you for using the word "apologist" so I didn't have to.)

That was really all I had: expressions of astonishment at the prevalence of Google apologist comments here today.

In an effort to be constructive let me say, it's a matter of scale. Google can't be fair because of their scale. Contrariwise, I use small-scale services and get outstanding customer service.

E.g. Tarsnap, cperciva is a god among men when it comes to customer service. Or sr.ht (Sourcehut) ddevault is right there on IRC most days. Or Podia, I wanted a certain font on my site, they didn't have it, I emailed them about it and they literally added it within the hour. Give these folks your money, eh?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#442

Post by rurp »

mcv wrote: post #312
paulgb wrote: post #298

Earlier quoted context omitted.

We were lucky enough that getting to the front page of HN did work for us[1], but it's worth noting that it doesn't even have to be your card ! Simply having a card from an issuer (in our case a legit, YC-backed, US-based startup bank) that got flagged was enough to have our account suspended and appeals denied. [1] https://news.ycombinator.com/item?id=32237445

Sounds like credit cards are the wrong payment system for GCP. Do they support direct bank transactions? They definitely should, if they don't trust credit cards (which I can understand).

Not to be glib but it sounds like GCP is the wrong cloud system to use in production. Credit card numbers get stolen and show up in weird places all the time, and the way Google handles these cases shows you what Google thinks of its customers. Why build on such an untrustworthy platform when there are alternatives?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#443

Post by happyopossum »

whoevercares wrote: post #407
luka-birsa wrote: post #386

A suggestion for engineers that lack business experience. As soon as your project grows beyond the non-hobby scale and starts making real money, invest some time to reach out to company providing the services that keep it alive and find an account manager that will take care of you. We're a Google customer and of course we had our fair share of issues (btw: it's the same with Azure, based on our experience), and we a…

That’s a good suggestion - it worth noting it comes with a cost (eg X% of your cloud bill) but you get much better interaction with the support and service team. AWS sometimes even connects small players with service team PMs and managers. It was really great but probably only for greenfield services

> it worth noting it comes with a cost (eg X% of your cloud bill) but you get much better interaction with the support and service team.

An account manager doesn't cost you anything - neither would a CE (or SE). There would likely be a minimum spend required, or if you're below that a commit, but there's no %age of annual spend charged for having an AM. In reality, and AM will save you money, as you can negotiate with them, whereas you can't with a website.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#444

Post by rurp »

hbn wrote: post #395
staticassertion wrote: post #326

Earlier quoted context omitted.

I hear way too many stories like this about Google products. Whether it's Youtube, AdWords, GCP, or anything else - you get blackholed by them and there's zero recourse. I just can't trust Google for something like running production services.

Several years back I wanted to try my hand at making an Android app and putting it in the Play Store. I knew ahead of time from reading all the horror stories that it's best to make a second dedicated developer Google account because once you turn an account into a developer account you're basically painting a target on your back to be randomly deleted by the powers that be. Additionally, not sure if this is still th…

The last time I tried to make a new Google account it required a valid phone number, so this might require getting a second phone line these days. I wouldn't trust Google to only disable one account if there is a related one using the same number.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#445

Post by upupandup »

Not only that they stringed me along for the free credit thing and I realized that I started to incur expenses for running some other instances. When I protested they assured me I billing will handle it. Except my account manager just ghosted me and I found out that if I do a chargeback all of my google services could be shut down, not just my GCP related billing.

I've never returned and have been with AWS since. Nobody in their right mind uses GCP and the only good thing they had was Firebase but already Supabase has pretty much rendered that useless.

Fck google cloud and I say that with confidence. If you are one of those shops that require engineers to use both GCP and AWS, start choosing, because new and old are on AWS and will remain on AWS.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#446

Post by aseipp »

MOARDONGZPLZ wrote: post #427
notyourday wrote: post #416

Earlier quoted context omitted.

I have dealt with Google and AWS under 4 different companies that spent between low double digit to high triple digit a month. General view: * AWS - Great customer service. Steak house level customer service when moved from cc billing to contract billing. * GCP - terrible service regardless of spend and regardless of dynamic or contract billing. AM do not know products. AM's are equivalent of BOA branch employees tha…

Same experiences here. AWS was awesome. They did crazy things I wouldn’t expect like spend days in person and join us at equinix colo to get the direct connect working. We didn’t even spend that much relatively, under $100k/mo.

At a prior gig we were using their FPGA instances and had a way of causing hard faults on the system that required reboots, with specific FPGA bitstreams. Their systems monitored this and when we reached out about it they scheduled at least 2-3 different calls with the engineers/PIs who were directly responsible so we could coordinate with them and help pin down the issue to get things working. We were literally renting by the hour; I don't even think we had an account manager, with variable costs; maybe a few hundred to 3k a month?

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#447

Post by upupandup »

imroot wrote: post #438
luka-birsa wrote: post #386

A suggestion for engineers that lack business experience. As soon as your project grows beyond the non-hobby scale and starts making real money, invest some time to reach out to company providing the services that keep it alive and find an account manager that will take care of you. We're a Google customer and of course we had our fair share of issues (btw: it's the same with Azure, based on our experience), and we a…

Ironically enough, the last time I was "Courted" by google cloud to move some of our infra there (after it went public that we had raised a round), they were up front about how their "account management sucks" and that we'd have a better "customer experience" if we went through a reseller. That...always sticks out to me, but, when dealing with the Google Education folks, there were some things that even resellers cou…

Thing is parent's advice is horrible because the account manager at google simply started to ignore my billing issues. Like I could reach out to no one, the account manager realized I was going to leave and stopped responding.

So the "make sure you invest in an account manager" is not really an option at Google. AWS does this very proactively.

Unfortunately after 5 years, I can still see Google Cloud screwing over their customers like this. They seem determined to send more customers to AWS.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#448

Post by soheil »

baobabKoodaa wrote: post #133
intellectronica wrote: post #115

Generally, all cloud providers have two ways of engaging with and billing customers. One is using a credit card, which is very easy for getting started. The other one is having an established relationship with a sales / account team and setting up billing by invoice. Smaller companies sometimes don't understand that and stick with credit card billing for too long. Once you move to production and have your own custome…

Ah yes, it was the customer's fault. Stupid customer using method recommended by service provider.

This is the line where HN becomes not so useful. Maybe try to imagine there are other professions beyond engineering that have existed in parallel that have their way doing things oh I don't know like following standards and best practices in accounting.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#449

Post by Sohcahtoa82 »

robertlagrant wrote: post #121
antihero wrote: post #56

Earlier quoted context omitted.

In these industries the regulation is the only thing preventing them being unfathomably worse.

It's not the only thing.

Market competition only goes so far.

Re: Tell HN: Google Cloud suspended our production projects at 1am on Saturday

#450

Post by upupandup »

MOARDONGZPLZ wrote: post #427
notyourday wrote: post #416

Earlier quoted context omitted.

I have dealt with Google and AWS under 4 different companies that spent between low double digit to high triple digit a month. General view: * AWS - Great customer service. Steak house level customer service when moved from cc billing to contract billing. * GCP - terrible service regardless of spend and regardless of dynamic or contract billing. AM do not know products. AM's are equivalent of BOA branch employees tha…

Same experiences here. AWS was awesome. They did crazy things I wouldn’t expect like spend days in person and join us at equinix colo to get the direct connect working. We didn’t even spend that much relatively, under $100k/mo.

THIS. I love how AWS AM go out of their way to deliver, where as Google's AM simply start to ghost you when the conversation gets to heated or not contributing to their bonus cheques.

As I've said and others have experienced, GCP's AM are killing their business and we've built up everything around AWS. There's no need to even learn Terraform, all of what we do are automated anyhow without it. CDK is enough.

Screw Google, screw Azure.

Post reply on HN