Live data from Hacker News

Ask HN: Shouldn't Google need to give a public statement about Railway incident?

news.ycombinator.com

11–20 of 116 posts

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#11
I think so. I'm a GCP user and I'm afraid of hosting workloads there now. I've heard too many nightmare stories, and I thought Google would be proper and thus not be infested with these kind of problems that cheaper providers are known for.

Maybe AWS is the only player in town now? I don't know. Google doesn't instill confidence with these incidents, same with those cases of insurmountable bills caused by simple mistakes where there should be a way for smaller customers to cap usage.

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#13
post #5

Would you want your vendors publicly disclosing potentially private reasons for an outage?

yes, especially since this didn't seem to be exactly "private" where it was anything specific, it was just some kind of automated system without a human in the loop

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#14
post #13
post #5

Would you want your vendors publicly disclosing potentially private reasons for an outage?

yes, especially since this didn't seem to be exactly "private" where it was anything specific, it was just some kind of automated system without a human in the loop

But shouldn't that be disclosed to Railway, and not the public? If they had someone running a botnet on compromised accounts there, for example.

If Railway isn't satisfied with the explanation, they're able to say so publicly, yes?

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#15
post #8
post #4

Yup, I think so. Makes one think about how dependent we are on cloud infra for core pieces versus supporting pieces of the architecture. They've probably negotiated some kind of private settlement.

I've directly asked our account manager about it. It's pretty scary that we don't know what automated mechanisms could just cut us off.

I really think Google underestimates the damage they've done to their reputation with these. These incidents are rare, but they're common enough where you can't trust them reliably anymore, and if that's the case, why would you pick them over other vendors?

it's not like they're the only provider, or even the #1

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#18
Being an advocate for GCP all these years, I can only say the earlier you get out of it, the safer it is for your business. All it takes is for their automated system to go haywire, and you can say bye bye to all your goodwill and customers. Go look at twitter how many customers are blaming railway. Founder had history getting screwed by GCP, yet still choose to depend on them.

You can't rely your business on GCP. Honestly, this is the most silly way to kill your own business.

For context, copied from my post 3 years ago.

March 10, 2023 | hide | past | favorite As a 4 years customer, our production severs have been suspended by Google Cloud because we didn't fill up some information on-time. Contacted support but they expect us to wait for 24-48 hours to get it resolved while all our servers are down. Anyone linked with someone powerful in google cloud can help?

======

- Running production on google cloud for 4 yrs with my startup. 100% legit SaaS business.

- Always pay bills on-time no issue. Good customer never open tickets, ask for help or what just quietly pay my bills each month.

- Our servers was abruptly suspended yesterday midnight and my whole business is now down for > 10hrs.

- We run a SaaS business that other ecommerce stores rely on and have hundreds of paying merchants.

- My customers have been grilling me and I don't feel gcloud's trust and safety team understand/care how urgent the issue is.

======

Why were our servers suspended? Because we didn't fill up information in time?

- See https://imgur.com/a/x0Y3RJl

- Apparently they dropped us an email 10 days back that I missed out

- Titled "Important Information Regarding Your Google Account" with no indication of suspension or what in title.

- Given the number of subprocessor "Important" emails they send it's too easy to miss out the email.

- 10 days gone by and our servers were abruptly shutdown with zero suspension notification or what.

- We've been paying $400-$700/mo for the past 4 yrs consistently and they shut us down because we didn't fill up some information?

When I tried to ask them to at least temporarily get our servers back while the verification is ongoing, I didn't get any answers.

Google Cloud have zero empathy for customers.

It's not like my account got suspended for fradulent issue or what. It's suspended because I didn't fill up some information on-time and they don't even allow me to temporarily reactivate my services or what. Especially when I had to wait for hours to get their team to verify my details before I can get my servers back.

You can't trust them with your business. Don't run any production stuffs with Google Cloud, ever.

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#19
The company I work for uses GCP and we preciously had intermittent CloudSQL connection errors for a few hours. We reached out and they resolved it after a day or so and said there was a minor incident but I don’t think it was ever publically reported.

Re: Ask HN: Shouldn't Google need to give a public statement about Railway incident?

#20

It depends on what you mean by ‘need’. If you mean they should for PR purposes, I probably agree. If you are saying they should be required to by law, then no I disagree.

I don't suspect anyone here is demanding laws be written requiring every single player in any SaaS outage to make a public statement immediately following an event like this. That's an odd thing to state your preference on.

But, given that this incident unjustly caused real damages to another company, I am pretty certain that Google will be required to make some sort of response to this, and if it ends up in the courts, it will be public.

Post reply on HN