Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident Report: May 19, 2026 – GCP Account Suspension

blog.railway.com

Post reply on HN
Print view View original

71–80 of 279 posts

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#71

Post by e40 »

dantillberg wrote: post #62

What drives Google to apply these actions so completely and immediately, versus a more deliberate approach, with notification and delay before action, manual review for paying customers, or a warning to resolve within X hours/days? Once or twice could be errors or bad implementation, but these can't explain away the pattern. It would seem that Google's counsel has deemed that whenever _____ is detected, the company m…

The problem is scale. Google uses automation and doesn't have the people to review the actions of that automation. I never worked at Google but this is the most obvious explanation from watching these things happen for years and years.

Please, someone that worked at Google, please comment.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#72

Post by sa46 »

Animats wrote: post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

Railway has an overwhelming incentive to pin the blame on Google. This report doesn't answer why Google suspended Railway's account.

I'd wait for more details before adjudicating.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#73

Post by xp84 »

tcdent wrote: post #61
apple4ever wrote: post #52

Earlier quoted context omitted.

Exactly this, which is the problem with all modern accounts. No person to talk to so you can understand what happened and maybe fix it.

They most definitely have a person to talk to. They're not the largest Google Cloud user by far, but they are large enough to have human account reps.

And those reps might not be told what the reason is.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#75

Post by mxuribe »

ibejoeb wrote: post #12

I've been getting serious, recently, about moving all my workloads to equipment that I control in datacenters with which I have professional relationships. It's less expensive, easier, and this kind of nonsense doesn't happen. These cloud providers need to step back and observe how terrible they've made these products. Footguns everywhere, pricing that is impossible to forecast or reason about, broken APIs, and autom…

> ...These cloud providers need to step back and observe how terrible they've made these products...

I doubt that will happen because none of them want to stop the money-making machine they have! And, if your thought after my comment is that all us techies are making a fuss, so the cloud providers and businesses using them will hear our cries and trigger a backlash...? I doubt that to...because some senior business leaders that i see are bent on listening more to management consultants as opposed to abalance of folks including their own internal experts...but, alas, maybe i'm just having too cynical a day today. :-)

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#76

Post by efdee »

wouldbecouldbe wrote: post #35
levkk wrote: post #14

Earlier quoted context omitted.

DigitalOcean. Seriously. They have been around a long long time and built a lot of the core infrastructure you rely on every day (e.g. Ceph).

I;ve had my share of VPS & Managed DB outages at DO, so they are also not faultless.

I've been with DO since checks mailbox 2014. Honestly never experienced an unannounced outage.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#77

Post by JCTheDenthog »

tjwebbnorfolk wrote: post #56
Animats wrote: post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

They have not explained WHY their account was suspended. That's the most important part, imo. Cloud Providers don't suspend entire accounts for no reason.

Unfortunately the cloud providers also rarely if ever tell you the reason.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#78

Post by Saris »

ryanSrich wrote: post #11

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

Maybe a VPS? Simple to manage and way cheaper.

But really any service (or even on-site hosting) can have downtime, if that's not acceptable then I suppose building/using a tool that can be distributed between multiple hosts located in different geographical areas is the best option.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#79

Post by JohnMakin »

shwetanshu21 wrote: post #47

This should be a warning to anyone running GCP. They suspend accounts left right and centre without even thinking about what they're doing. It seems like they use Gemini 3.1 Pro to run their production decisions. TK has a history of absolutely destroying the culture of the place like in OCI and has done something similar in GCP from what I've heard. GCP and Google are completely different entities with how they work.…

All google products work like this. Should never be used for anything critical.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#80

Post by nozzlegear »

ryanSrich wrote: post #11

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

[dead]
Post reply on HN