Live data from Hacker News

Twitter Completely Down

isup.me

131–140 of 194 posts

Re: Twitter Completely Down

#131
post #78

I think this is another good example of how we as an industry are still unable to adequately assess risk properly. I'm fairly certain that the higher-ups in Twitter weren't told "We have pretty good failover protection, but there is a small risk of catastrophic failure where everything will go completely down." Whoever was in charge of disaster recovery obviously didn't really understand the risk. Just like the recen…

Whoever was in charge of disaster recovery obviously didn't really understand the risk. That's not necessarily true. People don't die when twitter is down, and whatever twitter's business model actually is, I am not even sure there is a monetary penalty to them being down (unlike, say, Amazon being down which results in lost orders). They may have made the calculation that it was not cost effective engineering-wise t…

They definitely lose money - their ad model is based on engagement, so if tweets aren't seen/can't be acted on, money is lost.

Re: Twitter Completely Down

#132

I think this is another good example of how we as an industry are still unable to adequately assess risk properly. I'm fairly certain that the higher-ups in Twitter weren't told "We have pretty good failover protection, but there is a small risk of catastrophic failure where everything will go completely down." Whoever was in charge of disaster recovery obviously didn't really understand the risk. Just like the recen…

Accidents happen. (expanding my reply) No risk assessment in the world will stop a cage monkey from tripping over a pile of 1Us and falling onto the big red button. Figure out what your pain threshold is and live with it.

I know of a chap, called Terry, who became known as Total Network Terry. TNT. Because he accidently plugged the wrong cable into the wrong jack in the wrong cable cupboard on the wrong floor.

Just finding it took days to sort out, apparently.

Re: Twitter Completely Down

#133
post #89

Earlier quoted context omitted.

Oh, did they create a new messaging protocol that is universally accessible, redundant and easy to use? Something the vast majority of people couldn't care less about. Twitter has, in effect, created a new messaging protocol, in the broadest sense of it. It's accessible on your computer, your phone, even your TV if you try hard enough. It's integrated with hundreds of apps and sites. Technically speaking it isn't doi…

> It's accessible on your computer, your phone, even your TV if you try hard enough. Note the irony of this comment existing in a post about the service not being accessible, anywhere . Which wouldn't be the case had it been a protocol.

A massively decentralised protocol would not exactly be the model of reliability, either. I'm not even sure how that could possibly work in a service like Twitter.

Re: Twitter Completely Down

#134
I think that a lot of you guys are confusing "Disaster Recovery" with "Business Continuity".

Disaster Recovery is a reactive approach. It's what you do to get things back up AFTER a system or site has failed.

Business Continuity is a proactive approach. It's what you do to ensure that your critical services will remain viable whenever disaster occurs.

In the cases of Heroku, Amazon, Twitter, and many more, their Disaster Recovery strategies have been successful. The fact that they came back online without major data loss is proof of that. Their business continuity strategies, however, have been found wanting.

Re: Twitter Completely Down

#135
post #31

One great thing about this for me: it exposed a bug in my app which relied on twitter being up.

If Twitter never went down, would it be a bug?

I had a similar issue in one of my sites. It's an unnecessary dependency and should be corrected. Maybe not a bug, but a problem.

Re: Twitter Completely Down

#136

Earlier quoted context omitted.

"Incredibly useful" would imply that if you told me about Twitter, I wouldn't believe you. As much as I wish it were otherwise, it's not that hard to believe that someone invented IRC over HTTP and gave it a silly name.

People don't expect "incredible" to mean "not credible" anymore.

That's literally the most awesome thing I've read today, for all intensive purposes!

Re: Twitter Completely Down

#137
post #133

Earlier quoted context omitted.

> It's accessible on your computer, your phone, even your TV if you try hard enough. Note the irony of this comment existing in a post about the service not being accessible, anywhere . Which wouldn't be the case had it been a protocol.

A massively decentralised protocol would not exactly be the model of reliability, either. I'm not even sure how that could possibly work in a service like Twitter.

It would probably work like an IRC network. A network might involve hundreds of servers, but you only notice which server another person is using when there's a netsplit. Split users can reconnect using another server and rejoin the channel with only a slight interruption.

An IRC-styled Twitter would need some sort of synchronization service to handle twitsplits.

Re: Twitter Completely Down

#138
post #133

Earlier quoted context omitted.

> It's accessible on your computer, your phone, even your TV if you try hard enough. Note the irony of this comment existing in a post about the service not being accessible, anywhere . Which wouldn't be the case had it been a protocol.

A massively decentralised protocol would not exactly be the model of reliability, either. I'm not even sure how that could possibly work in a service like Twitter.

Has email ever been down, like, everywhere?

Re: Twitter Completely Down

#140
post #71
post #14

Earlier quoted context omitted.

Twitter Site Issue 17 seconds ago Users may be experiencing issues accessing Twitter. Our engineers are currently working to resolve the issue.

"Users may be experiencing difficulties accessing Twitter"? Weasel words are so annoying and unnecessary. "Twitter is down." would do.

"Twitter is down" means something very different from "Some people are having trouble accessing Twitter." My own traceroute results during the outage pointed to network problems, and I saw people saying they could access it.

It was unavailable to many people, not down.

Post reply on HN