Live data from Hacker News

Incident Report: May 19, 2026 – GCP Account Suspension

blog.railway.com

11–20 of 279 posts

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#11
Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following

1.) Vercel - having a bad month

2.) Supabase - having a bad month

3.) Railway - now having a bad month

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#12
I've been getting serious, recently, about moving all my workloads to equipment that I control in datacenters with which I have professional relationships. It's less expensive, easier, and this kind of nonsense doesn't happen. These cloud providers need to step back and observe how terrible they've made these products. Footguns everywhere, pricing that is impossible to forecast or reason about, broken APIs, and automated self destruction. Then you have third-party providers sitting on top of them, adding another layer of each antifeature. Crazy.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#13

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

Haven't used railway but my understanding is they are something similar to Heroku. Fly.io has been pretty great for tiny projects in that niche.

For Vercel if your nextjs site can be compiled statically you could probably throw it up on almost anything. We've self hosted before which is pretty straightforward but you lose a lot of the image optimization stuff unless you go deep into setting up open next.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#14

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

DigitalOcean. Seriously. They have been around a long long time and built a lot of the core infrastructure you rely on every day (e.g. Ceph).

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#16
> May 19, 22:10 UTC - Our automated monitoring detected API health check failures and paged our on-calls, who started investigating the issue.

> At 22:20 UTC on May 19, Google Cloud placed Railway’s production account into a suspended status incorrectly, as part of an automated action.

If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended?

The simplest explanation is just that one or the other of these timestamps is wrong, which wouldn't be a big deal. But if the timestamps aren't known with certainty, it seems very odd to include them in the writeup as though they are certain, even though they are very obviously inconsistent with each other.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#17

> May 19, 22:10 UTC - Our automated monitoring detected API health check failures and paged our on-calls, who started investigating the issue. > At 22:20 UTC on May 19, Google Cloud placed Railway’s production account into a suspended status incorrectly, as part of an automated action. If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended? The simplest explanation is…

> If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended?

Assuming the timestamps are accurate, Google probably started terminating resources while the account was not "suspended" and only completed that after all resources were disabled.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#18
post #2

Railway has not had the best month in the tech press have they? And in both cases it was an automated process belonging to some other party that put them there, damaging their reputation. I was going to talk to our google rep about their killing the Gemini cli but this is way more concerning.

Building on someone else's platform is always gonna be a risky move, and building a platform on top of someone else's platform is even riskier.

My company used to use a hosting provider that was basically AWS plus some extra guarantees. We just finished migrating onto regular AWS because they now offer what we need directly.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#19
post #17

> May 19, 22:10 UTC - Our automated monitoring detected API health check failures and paged our on-calls, who started investigating the issue. > At 22:20 UTC on May 19, Google Cloud placed Railway’s production account into a suspended status incorrectly, as part of an automated action. If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended? The simplest explanation is…

> If the timestamps are accurate, what was causing the errors 10 minutes before the account was suspended? Assuming the timestamps are accurate, Google probably started terminating resources while the account was not "suspended" and only completed that after all resources were disabled.

Or the account started doing something nefarious (assuming one of their customers as root cause, not railway itself) that started causing real problems and Google shut it down.

The problem with not having the data is that it’s easy to make assumptions.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#20
post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

Never could. Google might block your entire company because one of your workers did something nasty on their personal account, and their ban hammer is mighty and blocks all related accounts to the Nth degree
Post reply on HN