Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GCP Incidents

blog.railway.app

Post reply on HN
Print view View original

1–10 of 167 posts

Re: GCP Incidents

#3

Post by b112 »

esafak wrote: post #2

When they say moving are off Google Cloud services to bare metal, where do they plan to move?

My response to this, is that there are endless ways, and places to do this.

There are far more colos, people that will rent you a rack, and bandwidth, than VPS types. And you can rent servers too, instead of buying your own.

Colo is literally 10000x cheaper than many AWS deployments. I've seen million dollar bills drop to tens of thousands per year.

And of course, you can always deploy in house, in your own server room.

Re: GCP Incidents

#4

Post by 363082a9-58a7 »

I've had an experience with GCP that involved a very enterprise-y feature breaking in a way that clearly showed the feature never worked properly up until this point (aside from causing downtime when they tried to quietly fix it). GCP reps proceeded to remind everyone in the call in which they were supposed to explain what happened they were under NDA, because admitting to the above would've been a nightmare for regulated industries.

Re: GCP Incidents

#5

Post by [deleted] »

b112 wrote: post #3
esafak wrote: post #2

When they say moving are off Google Cloud services to bare metal, where do they plan to move?

My response to this, is that there are endless ways, and places to do this. There are far more colos, people that will rent you a rack, and bandwidth, than VPS types. And you can rent servers too, instead of buying your own. Colo is literally 10000x cheaper than many AWS deployments. I've seen million dollar bills drop to tens of thousands per year. And of course, you can always deploy in house, in your own server ro…

[deleted]

Re: GCP Incidents

#6

Post by Animats »

As someone who's into virtual worlds, and a user of Second Life, it's impressive to see how well those systems stay up. There hasn't been a total outage of Second Life in 5-10 years. Once Amazon's networking went down in a way that prevented new logins for a whole day, but existing logins remained. The 3D world, which has a lot of stuff going on even with no users around, continued to work. This is an extremely complex one of a kind system, and it just keeps cranking along. It's very distributed; one region (a 256x256m square) can crash and restart without taking down its neighbors. Users see the failed region as a square hole in the ground filled with water until the server restarts, which takes about two minutes. So outages are quite graceful. It's currently hosted on AWS, but it doesn't have to be.

What fails? The associated webcrap. The Marketplace, which is just a catalog and shopping cart. The forum system, which is outsourced to Invision, seems to fail several times a month. The messaging system, which is just a lightweight social network. The billing system. The outgoing payments system. Amazon's outgoing HTTPS proxy. All of those have failed several times in the last year. Even the JIRA system conked out once.

The quality of web software is underwhelming.

Re: GCP Incidents

#7

Post by bagels »

b112 wrote: post #3
esafak wrote: post #2

When they say moving are off Google Cloud services to bare metal, where do they plan to move?

My response to this, is that there are endless ways, and places to do this. There are far more colos, people that will rent you a rack, and bandwidth, than VPS types. And you can rent servers too, instead of buying your own. Colo is literally 10000x cheaper than many AWS deployments. I've seen million dollar bills drop to tens of thousands per year. And of course, you can always deploy in house, in your own server ro…

Sure, colo can be cheaper. What kind of infrastructure was this? Or was this just bandwidth bills?

Re: GCP Incidents

#8

Post by dilyevsky »

b112 wrote: post #3
esafak wrote: post #2

When they say moving are off Google Cloud services to bare metal, where do they plan to move?

My response to this, is that there are endless ways, and places to do this. There are far more colos, people that will rent you a rack, and bandwidth, than VPS types. And you can rent servers too, instead of buying your own. Colo is literally 10000x cheaper than many AWS deployments. I've seen million dollar bills drop to tens of thousands per year. And of course, you can always deploy in house, in your own server ro…

I’ve done some modeling and for our high cpu, high egress, medium storage multi-million $ a year deploy it was 70-90% lower cost than cloud when you factor in amortized cost of boxes, remote hands, transit etc. Pretty substantial but not 10000x ;)
Post reply on HN