Live data from Hacker News

Twitter Completely Down

isup.me

91–100 of 194 posts

Re: Twitter Completely Down

#91

Earlier quoted context omitted.

99,9% of people doesn't care at all about how technically innovative something is, they only care about how useful it is. And Twitter is incredibly useful.

"Incredibly useful" would imply that if you told me about Twitter, I wouldn't believe you. As much as I wish it were otherwise, it's not that hard to believe that someone invented IRC over HTTP and gave it a silly name.

People don't expect "incredible" to mean "not credible" anymore.

Re: Twitter Completely Down

#92

Always fun when you're developing against an API, and then have to perform a frantic investigation to work out if your latest code change broke everything ... or it's just the API endpoint itself.

Unit test ( as already mentioned) and build up static json/xml/whatever files to run your code against so that you can integration test your stuff without integration testing the 3rd party stuff.

Re: Twitter Completely Down

#93

x is down: http://blog.jgc.org/2010/10/x-is-down.html

That's a good blog post. But "x is down" is newsworthy for sufficiently large number of users of x and sufficently long downtime. That's because consumer experience with this or that online service influences the online service's reputation. A service with few users, or users who have lower expectations, can endure more downtime without loss of reputation than a service with many users who expect the service simply always to be available, at least as available as broadcast television or plain-old telephone service.

Re: Twitter Completely Down

#95

Earlier quoted context omitted.

I think it's only newsworthy in this space - where, as some of us are the ones responsible for these systems - we are trying to learn why this happened in order to prevent it. You know, learn from others mistakes and all.

I agree that learning from why Twitter was down will be interesting and when that story comes out I hope it will be high on Hacker News. But this sort of news story about something that's happening right now is symptomatic of the useless '24 hour news' cycle of noise.

It seems like your perspective is that HN exists for the links it points to. I read HN for the discussion, and this thread has many good discussions in it.

Re: Twitter Completely Down

#96

Earlier quoted context omitted.

I think it's only newsworthy in this space - where, as some of us are the ones responsible for these systems - we are trying to learn why this happened in order to prevent it. You know, learn from others mistakes and all.

I agree that learning from why Twitter was down will be interesting and when that story comes out I hope it will be high on Hacker News. But this sort of news story about something that's happening right now is symptomatic of the useless '24 hour news' cycle of noise.

> But this sort of news story about something that's happening right now is symptomatic of the useless '24 hour news' cycle of noise.

The single most important part of the internet is the immediate availability of news (to me, anyway). I've never heard anyone complain about that before; why do you think it's not worth knowing and talking about events as they happen? 'Twitter is down' isn't noise. Years and years ago there were stories about fire departments (SF I think?) that started using Twitter to send out fire notices. Here in Montreal, the police tweet very quickly and accurately about our daily student protests.

Twitter is extremely important to a huge number of people, and when it goes down, a site like HN definitely should be talking about it. It's big news and it's almost exclusively relevant while it's happening.

I love the internet.

Re: Twitter Completely Down

#98
post #63

I think this is another good example of how we as an industry are still unable to adequately assess risk properly. I'm fairly certain that the higher-ups in Twitter weren't told "We have pretty good failover protection, but there is a small risk of catastrophic failure where everything will go completely down." Whoever was in charge of disaster recovery obviously didn't really understand the risk. Just like the recen…

I'd guess the higher ups at Twitter have run the cost benefit in their head (and probably many spreadsheets) plenty of times, and in most cases spending your limited resource on disaster recovery preparation just isn't worth it. Their site being down does not qualify as a "disaster" - they'll be back up soon, then we'll all be tweeting away again within minutes.

I agree - risk assessment does not mean mitigate all risks 100%. Risk is a trade between cost and consequences of failure.

Re: Twitter Completely Down

#99

Earlier quoted context omitted.

99,9% of people doesn't care at all about how technically innovative something is, they only care about how useful it is. And Twitter is incredibly useful.

"Incredibly useful" would imply that if you told me about Twitter, I wouldn't believe you. As much as I wish it were otherwise, it's not that hard to believe that someone invented IRC over HTTP and gave it a silly name.

Christ, enough with the pedantry. I'm getting sick of all the pedants on HN critiquing idioms and common phrases.

Did you understand what the GP meant? Yes. Did you need to point out to the world your complete mastery of the English language? No.

The GP's point was that Twitter is useful, despite it's relatively low score on "innovative new technology" scale.

Re: Twitter Completely Down

#100

I think this is another good example of how we as an industry are still unable to adequately assess risk properly. I'm fairly certain that the higher-ups in Twitter weren't told "We have pretty good failover protection, but there is a small risk of catastrophic failure where everything will go completely down." Whoever was in charge of disaster recovery obviously didn't really understand the risk. Just like the recen…

This is a really good point.

Any assessment of risk entails certain larger assumptions about the world, many of which often turn out to be mere guesses.

Consider all the prices that are set to their current levels b/c nobody expects the collapse of the US political system to occur. Yet there is a nonzero probability that it will occur.

On one hand this seems like an absurd example, yet it exemplifies the kind of blind spot we are prone to when assessing risk. We generally address all the risks we can directly control, then classify the rest as "systemic" which essentially means that we are not able to compute them so we're going to ignore them.

Yet many systems which we assume to be stable or predictable (governments, companies, markets, weather patterns, social trends, etc.) have unexpected aberrations now and then which can have very significant consequences. Since these tend to impact most companies equally, the market will converge on an equilibrium where no firms do anything to hedge against these things.

Do you want to pay extra bank fees so that your bank can hedge against the collapse of the US currency for your checking account? Probably not. Do you want to triple your hosting costs to hedge against a massive US power grid failure? Probably not. The same applies to asteroid risk and sudden ice age risk.

On the other hand, if you have lots of money saved, you may wish to hedge against the collapse of one currency or another, and if your business would end if you suffered a few hours of downtime, you might want to invest in massive amounts of redundancy.

Every morning when we all commute to work we risk death. Some exposure to systemic risk is considered acceptable, and part of the character of any person or business is the kind of risk exposure we tolerate day to day. A doctor working in an AIDS clinic risks needle sticks and HIV, a startup doubling its users each month risks downtime but also risks a cash flow crisis.

Post reply on HN