Live data from Hacker News

Incident Report: May 19, 2026 – GCP Account Suspension

blog.railway.com

41–50 of 279 posts

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#41
post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

They trust them enough to still give them money, just goes to show how entrenched big tech is and why they need to be broken up into dozens of pieces.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#44
post #33

The interesting and yet-to-be-explained part is why google flagged the account? Put all the timestamps you want in the post mortem about what you observed, but you haven't addressed the root cause. The "this doesn't make sense" part of the story likely has a real explanation that nobody wants to reveal yet.

Shouldn't Google answer this if they are unhappy with this incident report? Are we even sure that Railway knows?

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#45

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

Hetzner (or any VM provider) + Dokku works best.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#46

> May 19, 22:10 UTC - Our automated monitoring detected API health check failures and paged our on-calls, who started investigating the issue. > At 22:20 UTC on May 19, Google Cloud placed Railway’s production account into a suspended status incorrectly, as part of an automated action. If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended? The simplest explanation is…

The 22:20 timestamp from the body of the post is wrong. The timeline section (where the 22:10 timestamp came from) is consistent with itself, and also contains:

> May 19, 22:19 UTC - Root cause identified: Google Cloud Platform has suspended Railway's production account.

They couldn't have identified the root cause before it happened.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#47
This should be a warning to anyone running GCP. They suspend accounts left right and centre without even thinking about what they're doing. It seems like they use Gemini 3.1 Pro to run their production decisions.

TK has a history of absolutely destroying the culture of the place like in OCI and has done something similar in GCP from what I've heard. GCP and Google are completely different entities with how they work. Don't expect Google quality from the name. It's just like those old brands which now have cheap licensed products like Nokia (An exaggeration I know but not far from truth).

Not only that they are known to shut off their services randomly giving you like 6 months to migrate. They have lots of engineers not doing anything, so they put them on migrating internal users off those services, most of their clients don't. There was a brilliant article on this by an ex-GCP employee that I can't find right now.

Avoid GCP like plague if you are serious about your business.

Edit: Gemini (unironically) found the article on this, a very good read: https://steve-yegge.medium.com/dear-google-cloud-your-deprec...

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#49
> Your customers don't care whether the failure was Google or Railway; they see your product.

Refreshing. So tired of businesses blaming their vendors. Oh it wasn't us spamming you text messages and emails, it was Shopify. Oh, our delivery guarantee said 2 days and it's been a week? That's not us, it's UPS.

I don't care. I didn't pay UPS or Shopify. I paid you.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#50
post #33

The interesting and yet-to-be-explained part is why google flagged the account? Put all the timestamps you want in the post mortem about what you observed, but you haven't addressed the root cause. The "this doesn't make sense" part of the story likely has a real explanation that nobody wants to reveal yet.

That‘s the point where Google tells you they won’t tell you the exact reason because of security reasons
Post reply on HN