Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident Report: May 19, 2026 – GCP Account Suspension

blog.railway.com

Post reply on HN
Print view View original

261–270 of 279 posts

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#261

Post by Twirrim »

keeda wrote: post #227
shwetanshu21 wrote: post #47

This should be a warning to anyone running GCP. They suspend accounts left right and centre without even thinking about what they're doing. It seems like they use Gemini 3.1 Pro to run their production decisions. TK has a history of absolutely destroying the culture of the place like in OCI and has done something similar in GCP from what I've heard. GCP and Google are completely different entities with how they work.…

At least from the outside TK seems to be doing well given GCP's growth. My completely uninformed assessment is that he stepped in as the disciplined adult in the room to override Google's otherwise lackadaisical approach to enterprise. (Clearly still some ways to go, as this incident shows.) Now, that may have created a culture that is at odds with the rest of Google, but it was probably required to become a "serious…

I've been part of OCI for 10 years now. TK did nothing to the culture in OCI. Virtually everything that OCI was and has become has come from its own leadership (tone very strongly set by both Don and Clay, in their own ways). Mostly we lost that initial scrappiness of "do it now, ask for forgiveness afterwards" that is inevitable when you shift from a business with no customers, to one with a large and growing customer base. You can't exactly just fling stuff to production and see what happens any more.

opinion my own, yada yada yada.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#262

Post by neya »

scott_w wrote: post #256
neya wrote: post #252

Earlier quoted context omitted.

Or they fixed the issue to remain compliant so GCP would restore access. Again, I know it's fashionable to hate on Google here, but there are always 2 sides to every story.

That’s not what you just claimed. I’ve quoted you below: > Or..they couldn't remain compliant with one of the strictest cloud vendors' policies. Which is it? They can comply or they can’t? Both sides can’t be true at the same time.

Just saying either way is possible unless both sides of the story are out in the open. You seem to start from a point of view of malice, I'm simply suggesting possible alternatives.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#263

Post by pocksuppet »

majdalsado wrote: post #34

Unfortunately we had to make emergency migration off to Azure yesterday due to this. Thankfully our DB was not hosted on Railway and we were back up in a couple hours. As much as we loved the simplicity they provided us, there's just been too many mishaps and shortcomings for us to continue running a B2B enterprise app on their infrastructure. Sad day :(

Railway can probably sue Google for their loss of income from you. Should be fun.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#265

Post by llmslave »

mjcl wrote: post #208
llmslave wrote: post #134

Earlier quoted context omitted.

For big accounts, like railway, zero chance this was a handsoff fully automated ban

Really? This isn't the first time their automation took down a big customer (UniSuper in 2024) by accident. In that case the automation actually deleted the resources and GCP had to recover them.

I have worked inside major cloud providers, it doesnt work like this

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#266

Post by pocksuppet »

neya wrote: post #229
shwetanshu21 wrote: post #47

This should be a warning to anyone running GCP. They suspend accounts left right and centre without even thinking about what they're doing. It seems like they use Gemini 3.1 Pro to run their production decisions. TK has a history of absolutely destroying the culture of the place like in OCI and has done something similar in GCP from what I've heard. GCP and Google are completely different entities with how they work.…

This is just typical anti-Google FUD. Been on GCP for nearly a decade and as do my peers. Sure, you hear about a few stories like these and in this particular case with Railway, I would actually wait to see what caused the trigger for the suspension - both from Railway and from GCP. But, this sort of thing happens with every cloud provider including AWS (you can google for the same thing "AWS shut down our account wi…

This is called "victim blaming" fwiw

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#267

Post by anon7000 »

skarz wrote: post #259
boylan wrote: post #105

Earlier quoted context omitted.

This exact thing happened to me when I ran https://www.fatherly.com/ circa 2017. Google just shut down our account without notice. We were spending like $10k/month. It also locked us out of our premium support account, so we couldn't even get anyone there to notice that they'd locked us out. After about 8 hours, a random Google support tech said it was because we were mining bitcoin, which was laughably untrue. We ha…

$10k a month on a blog??

Magazine

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#268

Post by Animats »

stackskipton wrote: post #207
londons_explore wrote: post #119

Earlier quoted context omitted.

My guess would be the credit card expired.... If it were something out of Railways hands, I think they would say something like "We have not yet identified the reason for the suspension, and are awaiting a response from Google".

At any company doing Enterprise work, you don't cut off someone for non payment without Account Manager doing multiple phone calls to whoever you have contact information for, emailing everyone listed on the account and whoever opened a support ticket and maybe even putting a banner in the panel with "ACCOUNT OVERDUE, CALL US TO SORT IT OUT!" Generally it takes 30 days past due and complete no contact for anyone befo…

That was normal US B2B behavior until recently. A supplier that cut off their customers for no good reason would lose their reputation in the industry and have real problems selling.

Then we got down to a very small number of major suppliers in some areas. Many have moats that deter switching. This gives the big suppliers too much power.

A century ago, this happened with railroads. Which is why railroads were forced to become common carriers. They're required to transport goods for anybody who can pay. Read up on antitrust activity of a century ago. We've been here before.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#269

Post by scott_w »

neya wrote: post #262
scott_w wrote: post #256

Earlier quoted context omitted.

That’s not what you just claimed. I’ve quoted you below: > Or..they couldn't remain compliant with one of the strictest cloud vendors' policies. Which is it? They can comply or they can’t? Both sides can’t be true at the same time.

Just saying either way is possible unless both sides of the story are out in the open. You seem to start from a point of view of malice, I'm simply suggesting possible alternatives.

Except one way isn’t possible because the facts and logic tell you it’s not.

Being open minded is good but don’t keep it so open your brain leaks out.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#270

Post by binarycleric »

pocksuppet wrote: post #263
majdalsado wrote: post #34

Unfortunately we had to make emergency migration off to Azure yesterday due to this. Thankfully our DB was not hosted on Railway and we were back up in a couple hours. As much as we loved the simplicity they provided us, there's just been too many mishaps and shortcomings for us to continue running a B2B enterprise app on their infrastructure. Sad day :(

Railway can probably sue Google for their loss of income from you. Should be fun.

Unlikely to be successful. GCP and AWS have pretty tight terms of service and you're legally SOL unless there was true negligence. Accounts rarely get suspended for no reason. There is likely more to this story that Railway isn't sharing.
Post reply on HN