I respect what railway is doing but also would never run my business on such a platform.
That kind of sounds like you don’t respect what they are doing.
Incident Report: Railway Blocked by Google Cloud [resolved]
31–40 of 381 posts
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#32I respect what railway is doing but also would never run my business on such a platform.
Today changed my opinion on them completely. Was willing to give them the benefit of the doubt that they're growing fast, but now seeing that they've failed to scale properly, and are missing little things that become big things later. I can't take that risk.
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#33so....what are we switching to y'all? cloud-run ? ;P
raspberry-pi cluster in my closet
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#34Wait… railway runs on GCP? Didn’t they make a whole thing about not “building a cloud on top of another cloud?” Or did they just mean that they’re not renting VPSs but only metal from the cloud provider? In my mind I was so excited that there was another provider not just paying one of the hyperscalars but at a minimum colocating and owning more of their stack. https://blog.railway.com/p/heroku-walked-railway-run
from the blog linked via Wayback Machine.
"From Day 1, we had this notion at the forefront.
The other notion that we have intuited is that you can’t build a cloud on another cloud. We have devoted years of practice running our own metal (and playing well with other clouds) to make sure that Railway’s business, which invariably becomes your customer’s business, is as rock solid as possible."
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#35TL;DR: putting all your eggs into one basket is bad, man.
That’s true, however having only few eggs and shopping for several baskets does not make sense in early days. Not sure how big railway is, but usually you start small with one egg.
You’d think they wouldn’t have started with GCP. There are plenty of datacenters where you can buy racks and racks of servers, and talk to a human when something goes wrong, and even walk in and access your servers. That’s what I’d be using if I were to build a Rackspace today.
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#36all my fkn postgres bd in railways! what do i do now?
Hahah at least you're not getting called every five minutes because you cant shut off the alerts, because its apparently deployed SOMEWHERE but good luck finding how to access it. Can't wait to see the bill from Twilio because of this lol
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#37Wait… railway runs on GCP? Didn’t they make a whole thing about not “building a cloud on top of another cloud?” Or did they just mean that they’re not renting VPSs but only metal from the cloud provider? In my mind I was so excited that there was another provider not just paying one of the hyperscalars but at a minimum colocating and owning more of their stack. https://blog.railway.com/p/heroku-walked-railway-run
Yep, and this is why I'm pissed. They lied. They're completely dependent on GCP. So, I gotta do some research, i need something a little more stable (and less dependent on one company's whims) than this. This is bad for them, because it really strikes at the heart of their 'big claim,' peacefull software deployments. This is chaos.
Yea, I mean, that's the whole MO of our platform and we failed at that. So yea, that's disappointing and more so for our customers.
I can provide an explanation about the GCP dependency. Yes, we have host workloads off GCP, and we have been able to build a good business by performing a cloud exit. However, we were worried that we would have a circular dependency on our own cloud. I don't think we expected to get auto-modded out of our own account, hence we left our DB on CloudSQL.
It was never our intent to deceive people that we didn't own our own destiny with our business. The last GCP issue, we were assured that this scenario wouldn't happen (when we got auto-ratelimited, which was bad, but survivable) - but it seems like we have further work to do. Apologies.
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#38Google cloud also locked out a Korean Goverment Organization recently. The guy posted on GCP subreddit. Google really need to improve their support team. It's strange such a big corp can't even afford to have proper support team.
> It's strange such a big corp can't even afford to have proper support team This seems to be by design.
We have a CSM, Head of Customer Support contact, and further contacts with GCP. Despite that, we still had this issue.
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#39It has been 0 days since GCP has taken down a startup (again).
You see this at least once a year. Never heard of this from AWS or Azure.
In all seriousness, this is why we don't use them. They have the most ergonomic cloud of the big three, then absolutely murder it by having this kind of reputation.
Re: Incident Report: Railway Blocked by Google Cloud [resolved]
#40Earlier quoted context omitted.
That’s true, however having only few eggs and shopping for several baskets does not make sense in early days. Not sure how big railway is, but usually you start small with one egg.
You’d think they wouldn’t have started with GCP. There are plenty of datacenters where you can buy racks and racks of servers, and talk to a human when something goes wrong, and even walk in and access your servers. That’s what I’d be using if I were to build a Rackspace today.
They started on GCP and have been migrating to their own "Metal" DC doing exactly what you're describing. But GCP is still their overflow given how rapidly they are growing and holds some amount of networking that routes to their DC.