Live data from Hacker News

Incident Report: Railway Blocked by Google Cloud [resolved]

status.railway.com

171–180 of 381 posts

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#171
post #160

The 3-2-1 backup rule is pretty outdated in the world of cloud. You could have 3 complete copies of your data in different S3 buckets, but if they're all under the same account you've lost your blast radius protection

If only there were a quick and easy way to replicate s3 buckets to an independent provider… … on the Unix command line … … to a cloud older than AWS… … if only …

I don't think that technology exists. Sorry.

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#172
post #160

The 3-2-1 backup rule is pretty outdated in the world of cloud. You could have 3 complete copies of your data in different S3 buckets, but if they're all under the same account you've lost your blast radius protection

If only there were a quick and easy way to replicate s3 buckets to an independent provider… … on the Unix command line … … to a cloud older than AWS… … if only …

Well having backups help, but I certainly can’t migrate my infra to rsync.net on moments’ notice (or ever since rsync.net does storage and nothing else) so my customers aren’t affected.

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#173
post #160

The 3-2-1 backup rule is pretty outdated in the world of cloud. You could have 3 complete copies of your data in different S3 buckets, but if they're all under the same account you've lost your blast radius protection

If only there were a quick and easy way to replicate s3 buckets to an independent provider… … on the Unix command line … … to a cloud older than AWS… … if only …

Wish I could upvote this comment account more. Too many people look for something new and shiny when trusty ol tools are sitting right there. :)

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#174

It has been 0 days since GCP has taken down a startup (again). You see this at least once a year. Never heard of this from AWS or Azure. In all seriousness, this is why we don't use them. They have the most ergonomic cloud of the big three, then absolutely murder it by having this kind of reputation.

On the other hand i can’t remember when there was a serious outage on GCP, unlike AWS/Azure who seem to go down catastrophically a couple of times per year.

Perhaps you don't notice GCP outages because so few companies rely on them?

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#176
post #31

I respect what railway is doing but also would never run my business on such a platform.

That kind of sounds like you don’t respect what they are doing.

I think it’s good people are making IaaS platforms, but have dealt with enough firefighter hero bullshit to have seen this coming a mile away. Uptime and redundancy are strongly correlated.

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#177

It has been 0 days since GCP has taken down a startup (again). You see this at least once a year. Never heard of this from AWS or Azure. In all seriousness, this is why we don't use them. They have the most ergonomic cloud of the big three, then absolutely murder it by having this kind of reputation.

On the other hand i can’t remember when there was a serious outage on GCP, unlike AWS/Azure who seem to go down catastrophically a couple of times per year.

How is blackhole-ing a customer not considered an outage?

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#178

As someone who runs some public APIs, the amount of spam from Railway IPs is insane. They have horrible abuse prevention. Hopefully this encourages them to improve their operations.

This is the conflict at the center of running a hosting company - make it easy to signup and you get a lot of new users but also a lot of abuse.

Implement anti-abuse measures and you will hit some loud false positives (this may be the case with GCP here).

I don't envy anybody running a hosting co - the internet is a really ugly place under the surface.

edit: to add - AWS are really good here. Must be the ~30 years of retail fraud and abuse experience.

Re: Incident Report: Railway Blocked by Google Cloud [resolved]

#179

It has been 0 days since GCP has taken down a startup (again). You see this at least once a year. Never heard of this from AWS or Azure. In all seriousness, this is why we don't use them. They have the most ergonomic cloud of the big three, then absolutely murder it by having this kind of reputation.

> Never heard of this from AWS or Azure. AWS does it more efficiently; it takes down many startups at a time when us-east-1 goes down.

And we all celebrate it since we can't do any work
Post reply on HN