Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident Report: May 19, 2026 – GCP Account Suspension

blog.railway.com

Post reply on HN
Print view View original

211–220 of 279 posts

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#211

Post by Citizen_Lame »

lightedman wrote: post #198
Citizen_Lame wrote: post #169

Earlier quoted context omitted.

There is always one bootlicker, fresh 1 day account no less.

Then let me be the not day-one account to say Railway is utterly bearing some responsibility here. "However, in this ring, there was still a hard dependency on workload discoverability being tied to the network control plane API that was hosted on the machines running in Google Cloud." They've gotta be joking me that they deliberately left something so critical under the control of any other entity than themselves. T…

There is always responsibility with Railway, that's given. But also taking into account how many big websites went down when AWS was down, building critical redundancy at such large scale is not cheap, and not many companies do it. Same as security theatre, we have redundancy theatre because they needed to sell the CLOUD.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#212

Post by [deleted] »

tcdent wrote: post #33

The interesting and yet-to-be-explained part is why google flagged the account? Put all the timestamps you want in the post mortem about what you observed, but you haven't addressed the root cause. The "this doesn't make sense" part of the story likely has a real explanation that nobody wants to reveal yet.

[deleted]

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#214

Post by nathancahill »

nightpool wrote: post #28
ryanSrich wrote: post #11

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

Fly, Render, and even Heroku still are all better choices then working with Railway I think

I love Fly, but their docs are.. tough. They've had multiple iterations of the control plane API, and it's very hard to do things the "correct" way with conflicting official docs.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#216

Post by tantalor »

Animats wrote: post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

Somebody else's computer

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#217

Post by guluarte »

praptak wrote: post #151
guluarte wrote: post #107

Earlier quoted context omitted.

This feels like google applying the same anti-spam mindset everywhere: detect risk, ban first, ask questions later.

It's pretty stupid that big customers like Railway are not somehow protected from this.

I think all customers should be protected by at least one CSR doing a quick approve before banning the account.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#218

Post by guluarte »

shwetanshu21 wrote: post #147
guluarte wrote: post #107

Earlier quoted context omitted.

This feels like google applying the same anti-spam mindset everywhere: detect risk, ban first, ask questions later.

That seems to be the case. But as we see it backfires. Railway is very public, but we know at hacker news google has been doing this kind of thing for quite some time now.

yep, last year they deleted UniSuper private cloud too

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#219

Post by dpkirchner »

nikanj wrote: post #20
Animats wrote: post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

Never could. Google might block your entire company because one of your workers did something nasty on their personal account, and their ban hammer is mighty and blocks all related accounts to the Nth degree

I've been wondering if I should be interrogating my friends before allowing them access to my wifi. "Have you or any of your family members ever been banned by Google?"

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#220

Post by dpkirchner »

charcircuit wrote: post #178

>Google Cloud placed Railway’s production account into a suspended status incorrectly, as part of an automated action. There is no justification given on why this action was incorrect. It's possible they actually did something wrong.

But not so wrong that they could get their account back in 10 minutes.
Post reply on HN