Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Incident Report: Railway Blocked by Google Cloud [resolved]

status.railway.com

Post reply on HN
Print view View original

211–220 of 381 posts

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#213

Post by shye »

garciasn wrote: post #106
binarycleric wrote: post #97

How the heck do these things happen, especially with companies with huge monthly spend? At my last job we had some suspicious workloads running on AWS and our TAM reached out to us before taking any action. Who wants to bet this was some AI automation gone wrong and because GCP seems to be allergic to actually contacting a human to get a response, this just sits in some support queue that outsourced workers look at a…

Nothing surprises me with anything related to support on GCP. While we absolutely do not need them, I have been through no less than 12 different Account Executives over the last 6y and they're all ENTIRELY and COMPLETELY useless. They all introduce themselves, beg me to setup a meeting w/them and some sort of engineering resource(s), and they come to a meeting with a canned slide deck that is so absurdly unrelated t…

That's exactly why I'm less pleased with GCP: to trust a CSP (or any service), I need to be assured that when (not if) things go wrong, I could escalate to a team that would have my back.

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#214

Post by Imustaskforhelp »

fragmede wrote: post #189
adamtaylor_13 wrote: post #174

Earlier quoted context omitted.

Perhaps you don't notice GCP outages because so few companies rely on them?

GCP has a lot of customers. But you wouldn't know the companies that do, unless you worked there and wanted to leak it, or it publicly comes out. Eg it's been publicly acknowledged that Apple uses GCP for iCloud, https://www.cnbc.com/amp/2018/02/26/apple-confirms-it-uses-g... , and Home Depot is another that's used as a case study, https://cloud.google.com/customers/the-home-depot but most customers don't want to mak…

upvoted & favourited because you taught me a really interesting fact which I feel makes up for an amazing discussion (regarding icloud using GCP).

also, I can't help but imagine if instead of render, it was Apple's account which could've been auto-banned (Render is almost a billion dollar company or series-B, I am not sure)

I haven't read the articles and I admit that but can you please elaborate to me on why Apple uses GCP themselves for idrive, I would love to know the technical decisions behind it on a genuinely curious level.

From my (let's face it) limited understanding of GCP, it isn't particularly good or price performant and one of the wonders is that Google sells it directly with Google photos too and an competitive lineup at android.

So in some sense if Apple is using gcp's for icloud then aren't they just reselling google storage themselves and google can always beat them in pricing while also wanting to chew away at the percentage of iphones themselves too?

I mean, I can still try to understand the google search pays apple 10 billion dollars (right?) deal but I don't quite understand why apple would pick GCP when the hosting market is one of the more competitive ones with lots of companies.

I would love to get some explainations or theories as to why exactly is that the case

(Also given its HN, if anyone from apple is reading or knows the answer, I would love that too!)

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#215

Post by erikerikson »

hasyimibhar wrote: post #206
erikerikson wrote: post #192

Earlier quoted context omitted.

The impacts are usually partial. For example, scaling is impacted but everything already deployed contributes to work up to capacity. Or, you can't change configuration but the previous configuration works as configured. Often surprisingly not so impactful even if there can be limited work stoppage.

The problem with the us-east-1 outage is that a lot of big companies are there, so even if you try your best not to depend on us-east-1, your third party providers are most likely there. In my previous company, we were completely down during us-east-1 outage because of other dependencies that are beyond our control.

Entirely fair. I have thus far avoided that problem. Not always engineering's choice.

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#217

Post by dlcarrier »

miniman1337 wrote: post #34
fjni wrote: post #24

Wait… railway runs on GCP? Didn’t they make a whole thing about not “building a cloud on top of another cloud?” Or did they just mean that they’re not renting VPSs but only metal from the cloud provider? In my mind I was so excited that there was another provider not just paying one of the hyperscalars but at a minimum colocating and owning more of their stack. https://blog.railway.com/p/heroku-walked-railway-run

from the blog linked via Wayback Machine. "From Day 1, we had this notion at the forefront. The other notion that we have intuited is that you can’t build a cloud on another cloud. We have devoted years of practice running our own metal (and playing well with other clouds) to make sure that Railway’s business, which invariably becomes your customer’s business, is as rock solid as possible."

I'm not familiar with Railway, so this might not make any sense, but it's possible they were using their own hardware but managing it with Google accounts. It's not uncommon for a company's offsite human-to-human communications to fail when there's a Google outage or ban, so it's not unexpected to have the same interference with human-to-machine or machine-to-machine communications.
Post reply on HN