Live data from Hacker News

Incident Report: May 19, 2026 – GCP Account Suspension

blog.railway.com

131–140 of 279 posts

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#131
post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

They have not explained WHY their account was suspended. That's the most important part, imo. Cloud Providers don't suspend entire accounts for no reason.

Google has suspended entire accounts countless times for absolutely no reason.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#133
post #33

The interesting and yet-to-be-explained part is why google flagged the account? Put all the timestamps you want in the post mortem about what you observed, but you haven't addressed the root cause. The "this doesn't make sense" part of the story likely has a real explanation that nobody wants to reveal yet.

Shouldn't Google answer this if they are unhappy with this incident report? Are we even sure that Railway knows?

the Railway PM doesn't say they weren't told. It just sort of glosses over this. I would be interested to know if they were told (or not).

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#134
post #33

The interesting and yet-to-be-explained part is why google flagged the account? Put all the timestamps you want in the post mortem about what you observed, but you haven't addressed the root cause. The "this doesn't make sense" part of the story likely has a real explanation that nobody wants to reveal yet.

I don't think you're typically told why for these things, and it's mostly automated from what I can tell. The automated systems make mistakes but more importantly they're completely opaque. Nobody, not even Google, knows how they work exactly.

For big accounts, like railway, zero chance this was a handsoff fully automated ban

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#135
post #15

"Finally, we are in planning to remove Google Cloud services from our data plane’s hot path, and keeping them only for secondary/failover." That's pretty clear. Google can no longer be trusted as a B2B service provider.

Hasn't every cloud provider had issues? Is the enshitification of servces really isolated to Google, or are we all doomed.

Banning accounts for no stated reason is kind of a Google speciality. They have a long well documented history of this sort of thing.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#136

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

Hatchbox + Digital Ocean is an unbeatable combo and provides Railway-like automation with self-owned infra.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#137
post #14

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

DigitalOcean. Seriously. They have been around a long long time and built a lot of the core infrastructure you rely on every day (e.g. Ceph).

Yes, I use DO with Hatchbox. It is a perfect combo. Been using for more and more projects.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#138
post #14

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

DigitalOcean. Seriously. They have been around a long long time and built a lot of the core infrastructure you rely on every day (e.g. Ceph).

I've had nothing but good experiences with them and their docs and tutorials are excellent.

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#139
post #115
post #105

Earlier quoted context omitted.

This exact thing happened to me when I ran https://www.fatherly.com/ circa 2017. Google just shut down our account without notice. We were spending like $10k/month. It also locked us out of our premium support account, so we couldn't even get anyone there to notice that they'd locked us out. After about 8 hours, a random Google support tech said it was because we were mining bitcoin, which was laughably untrue. We ha…

Google thinks everything should be replaced with automation. Remember knowledge cards? Prior to the LLM AI revolution, they had an extraordinarily crappy AI system digest the entire internet to figure out the wrong facts about stuff and then present it to users as solid truth, with no human review and no way to report inaccuracies. They just don't care. If the task requires a person to look at a thing and tell if it'…

Google is the epitome of expecting happy path results to always be the end result. I could absolutely see someone writing this knowledge card system, but then realizing how much work it would be to edit it with some PM not wanting to say the project was a failure and needing serious amounts of human effort to correct and just releasing it as is. Gotta earn those KPIs for that next promotion, and then it's someone else's problem!

Re: Incident Report: May 19, 2026 – GCP Account Suspension

#140

Question: for a smaller SaaS tool, or even internal product. If a team doesn't want to manage AWS or another IaaS provider, what are the best alternatives for the following 1.) Vercel - having a bad month 2.) Supabase - having a bad month 3.) Railway - now having a bad month

If you are unable to use IaaS directly. You need to accept that your service might be down. Even if you use AWS and the like, if you aren't building your app with redundancy across multiple AZs, then you'll have some downtime occasionally. And even if you do build redundancy with multiple AZ, some services might fail anyway as AWS is not entirely isolated. So you might have downtimes. So just accept downtimes and use…

The advantage of going with AWS is that when us-east-1 goes down, half the internet goes down so you don't have to defend why you had a service outage.
Post reply on HN