Linode Fremont servers are down

#1

Post by takinola » Thu, Jun 21, 2018, 5:10 AM UTC

Linode Fremont servers are down
status.linode.com

Re: Linode Fremont servers are down

#2

Post by empressplay » Thu, Jun 21, 2018, 5:21 AM UTC

Really dependable until they aren't =(

Re: Linode Fremont servers are down

#3

Post by taylorbuley » Thu, Jun 21, 2018, 5:22 AM UTC

empressplay wrote: post #2

Really dependable until they aren't =(

I usually take these in stride. I pay for cheap VPS, and sometimes I pay the price.

Re: Linode Fremont servers are down

#4

Post by Jack000 » Thu, Jun 21, 2018, 5:23 AM UTC

the last outage was in Fremont as well. I think I'll be moving my servers to Dallas or another provider..

Re: Linode Fremont servers are down

#5

Post by keyle » Thu, Jun 21, 2018, 5:24 AM UTC

Does it really take over an hour just to figure out why a whole data centre went dark?

Re: Linode Fremont servers are down

#6

Post by BluSyn » Thu, Jun 21, 2018, 5:26 AM UTC

Networking outage again. Sigh. So glad I just moved my prod servers to gcloud last week...

To be fair to Linode, the Fremont outages always seem to be caused by HE.net (the "upstream provider"). Used to co-lo in the same datacenter and had issues like this constantly. Not sure what HE's problem is.

Re: Linode Fremont servers are down

#7

Post by elliotlarson » Thu, Jun 21, 2018, 5:26 AM UTC

I've had the most problems with Linode Freemont historically. About 4 years ago I moved most of my stuff to Dallas and have been happy.

Re: Linode Fremont servers are down

#8

Post by LeoPanthera » Thu, Jun 21, 2018, 5:32 AM UTC

Looks like it's just come back, one of my servers just woke up.

Re: Linode Fremont servers are down

#9

Post by lsc » Thu, Jun 21, 2018, 5:33 AM UTC

BluSyn wrote: post #6

Networking outage again. Sigh. So glad I just moved my prod servers to gcloud last week... To be fair to Linode, the Fremont outages always seem to be caused by HE.net (the "upstream provider"). Used to co-lo in the same datacenter and had issues like this constantly. Not sure what HE's problem is.

The right solution is to get two upstreams and BGP.

...

which is unfortunately rather more difficult when you are in a datacenter owned by he.net :( It's possible, though, and I thought linode had done so, I think XO communications is in that building (that could be the other fremont he.net data center, it's been a while. Who owns XO these days? Verizon. I have been out for a while.)

I was in that same datacenter for a while, and I left due to power outages rather than network issues. (this was... man, almost a decade back now; most of that power equipment should have aged out by now.) - but he.net is a budget provider; I used them in my bgp mix for a lot longer...and usually they are fine. But I wouldn't use them without a backup. (he.net and cogent or whatever other cheap bandwidth is available usually is cheaper, and in aggregate, more reliable than a top tier provider on it's own.)

Re: Linode Fremont servers are down

#10

Post by madaxe_again » Thu, Jun 21, 2018, 5:35 AM UTC

This is kinda what a status page is for, not the HN front page - linked being down is nothing new - it’s why we moved our stack away to AWS, what, seven years ago? Well, that and the horribly overloaded switches and poor implementation of the Xen packet scheduler that caused unacceptable latency spikes to memcached.

It’s great for hosting toy sites that sit on a single box and don’t have an uptime SLA - and people saying “well it’s cheap” - moving to AWS saved money, as we had to run big linodes for the workload purely for the additional internode bandwidth.