stories like this are why i self-host most things on proxmox instead of depending on a single cloud provider. Ok I have to do the maintenance but at least this way no one can suspend my entire stack with not even an explanation.
Incident Report: May 19, 2026 – GCP Account Suspension
271–279 of 279 posts
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#272Earlier quoted context omitted.
They are a pension fund; they literally had/have US$125 billion dollars under management. What exactly is being stretched here? I can't for the life of me think of something that qualifies more for being a 125b company than actually having 125b in assets .
> I can't for the life of me think of something that qualifies more for being a 125b company than actually having 125b in assets. Which this company didn't. They managed 125b of assets belonging to other people, they didn't have 125b of their own.
Another solid C in finance. Congrats.
They have 125b they can literally write a check against and allocate any way they want as long as it delivers an adequate return and doesn't piss off the shareholders. Other than sophistry, what's the difference?
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#273Earlier quoted context omitted.
They are a pension fund; they literally had/have US$125 billion dollars under management. What exactly is being stretched here? I can't for the life of me think of something that qualifies more for being a 125b company than actually having 125b in assets .
Having assets under management doesn't mean you have that money. You don't own it, you are just taking care of it for somebody. When describing a company as an $X billion company, conventionally this is referring to the market cap. You could use it to describe other things they possess if you wanted to, but assets they manage will never be something they possess.
Ok, so we're engaging in sophistry. Got it.
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#274Earlier quoted context omitted.
They are a pension fund; they literally had/have US$125 billion dollars under management. What exactly is being stretched here? I can't for the life of me think of something that qualifies more for being a 125b company than actually having 125b in assets .
Companies are described by revenue. UniSuper made $110 million recently. It deceptive to use the assets managed as the size since it makes it look like a much larger company. NVIDIA has revenue of $130 billion. $125 billion revenue would make it the largest company in Australia by a good amount.
That's fair, but if I'm a fund that can literally write a check the size of my endowment, why is that not a demonstration of my value? That's not finance 101, but you get to it later on.
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#275Earlier quoted context omitted.
Having assets under management doesn't mean you have that money. You don't own it, you are just taking care of it for somebody. When describing a company as an $X billion company, conventionally this is referring to the market cap. You could use it to describe other things they possess if you wanted to, but assets they manage will never be something they possess.
Ok, so we're engaging in sophistry. Got it.
Language is a communication tool. If you misuse language you will be badly understood. The solution is to use the correct word for what you mean, not to accuse others of sophistry.
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#276Earlier quoted context omitted.
Ok, so we're engaging in sophistry. Got it.
Language is a communication tool. If you misuse language you will be badly understood. The solution is to use the correct word for what you mean, not to accuse others of sophistry.
How about 'pedantry'. I'd go with 'condescending', but that was clearly your intent.
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#277Earlier quoted context omitted.
Language is a communication tool. If you misuse language you will be badly understood. The solution is to use the correct word for what you mean, not to accuse others of sophistry.
How about 'pedantry'. I'd go with 'condescending', but that was clearly your intent.
People misunderstood your original reply. Correcting you is not pedantry.
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#278Earlier quoted context omitted.
> I can't for the life of me think of something that qualifies more for being a 125b company than actually having 125b in assets. Which this company didn't. They managed 125b of assets belonging to other people, they didn't have 125b of their own.
Another solid C in finance. Congrats. They have 125b they can literally write a check against and allocate any way they want as long as it delivers an adequate return and doesn't piss off the shareholders. Other than sophistry, what's the difference?
> They have 125b they can literally write a check against and allocate any way they want as long as it delivers an adequate return and doesn't piss off the shareholders.
No they don't. The pension regulator sets very restrictive controls on what they can do with that 125b, because it's neither the company's nor the shareholders' money.
Re: Incident Report: May 19, 2026 – GCP Account Suspension
#279Earlier quoted context omitted.
And this is Railway, a big enough name to top the HN main page and presumably find someone from Google to intervene at some point. I would have zero recourse if it was some little product that I built.
Their account was restored in 10 / 19 minutes! It just took 4-6 hours to get everything fully healthy. I look forward to seeing the google response to this hopefully. May 19, 22:10 UTC - Our automated monitoring detected API health check failures and paged our on-calls, who started investigating the issue. May 19, 22:11 UTC - Dashboard returning 503 errors. Users unable to log in. May 19, 22:19 UTC - Root cause ident…
The timestamp inconsistency teraflop points out is
interesting — but the bigger takeaway for me is that
Railway's own automated API health checks caught the
failure at 22:10, a full 10 minutes before the root
cause was identified.
That's external dependency monitoring working exactly as it should. Most teams only monitor their own infrastructure. When a cloud provider, payment gateway, or third-party API fails — your own dashboards show green while users see failures.
The lesson isn't specific to GCP — it's that monitoring what you depend on but don't control is just as important as monitoring what you own.