Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Twitter Completely Down

isup.me

Post reply on HN
Print view View original

121–130 of 194 posts

Re: Twitter Completely Down

#121

Post by debacle »

steve8918 wrote: post #47

I think this is another good example of how we as an industry are still unable to adequately assess risk properly. I'm fairly certain that the higher-ups in Twitter weren't told "We have pretty good failover protection, but there is a small risk of catastrophic failure where everything will go completely down." Whoever was in charge of disaster recovery obviously didn't really understand the risk. Just like the recen…

Good disaster recovery doesn't cost a ton of money, but it does require testing.

In almost every disaster situation I've been a part of, the UPSes have failed. Almost every time.

Disaster recovery has a horrible track record.

Re: Twitter Completely Down

#123

Post by timdorr »

Retric wrote: post #113
DanielN wrote: post #92

Earlier quoted context omitted.

Unit test ( as already mentioned) and build up static json/xml/whatever files to run your code against so that you can integration test your stuff without integration testing the 3rd party stuff.

And hope your static files still match the API behavior when your done...

That's actually a completely valid test to run: what happens when the API's results change on you? You should test that.

Re: Twitter Completely Down

#124

Post by rhizome »

hendler wrote: post #87
MattRogish wrote: post #32

I just ate lunch. It was sooo good! [Checked in to Shake Shack, NYC]

LOL @MattRogish RT I just ate lunch. It was sooo good! [Checked in to Shake Shack, NYC]

yep @hendler LOL @MattRogish RT I just ate lunch. It was sooo good! [Checked in to Shake Shack, NYC]

Re: Twitter Completely Down

#125

Post by rhizome »

steve8918 wrote: post #47

I think this is another good example of how we as an industry are still unable to adequately assess risk properly. I'm fairly certain that the higher-ups in Twitter weren't told "We have pretty good failover protection, but there is a small risk of catastrophic failure where everything will go completely down." Whoever was in charge of disaster recovery obviously didn't really understand the risk. Just like the recen…

That's not really fair, we as humans are bad at risk analysis. Bruce Schneier has written extensively on the role of cognitive biases et al in measuring perceived risk.

http://www.schneier.com/blog/archives/2009/08/risk_intuition...

Googling "Schneier risk" gets you lots and lots of reading material.

The only question here is whether Arrington is going to write another Amateur Hour post about this.

http://techcrunch.com/2008/04/23/amateur-hour-over-at-twitte...

Re: Twitter Completely Down

#126

Post by auxbuss »

mootothemax wrote: post #43

Always fun when you're developing against an API, and then have to perform a frantic investigation to work out if your latest code change broke everything ... or it's just the API endpoint itself.

In ruby, at least, there are some good tools to automate this for you. Using, say, vcr, you can automatically "record" your API calls (into "cassettes") as your tests run; these data are then used when you run your unit tests subsequently. When you plan to integrate/push, simply delete the "cassettes" and run your tests again. That way, any API changes are picked up prior to integration.

I have a suite here that takes about 3 minutes to run from scratch, but just over 1 second as unit tests.

Re: Twitter Completely Down

#128

Post by MatthewPhillips »

kristofferR wrote: post #54
k3n wrote: post #45

Earlier quoted context omitted.

Conversely, this has always been my complaint against the general hoopla and bandwagonning around Twitter -- they didn't do anything technically innovative. When they first started generating buzz, I instantly thought, "Oh, did they create a new messaging protocol that is universally accessible, redundant and easy to use? That's killer!" But, that was not the case. Yes, they had a good idea and executed very well, bu…

99,9% of people doesn't care at all about how technically innovative something is, they only care about how useful it is. And Twitter is incredibly useful.

Well, it's not incredibly useful right now, and it would be had it been a single node and not the entire damned thing.

Re: Twitter Completely Down

#130

Post by MatthewPhillips »

untog wrote: post #89
k3n wrote: post #45

Earlier quoted context omitted.

Conversely, this has always been my complaint against the general hoopla and bandwagonning around Twitter -- they didn't do anything technically innovative. When they first started generating buzz, I instantly thought, "Oh, did they create a new messaging protocol that is universally accessible, redundant and easy to use? That's killer!" But, that was not the case. Yes, they had a good idea and executed very well, bu…

Oh, did they create a new messaging protocol that is universally accessible, redundant and easy to use? Something the vast majority of people couldn't care less about. Twitter has, in effect, created a new messaging protocol, in the broadest sense of it. It's accessible on your computer, your phone, even your TV if you try hard enough. It's integrated with hundreds of apps and sites. Technically speaking it isn't doi…

> It's accessible on your computer, your phone, even your TV if you try hard enough.

Note the irony of this comment existing in a post about the service not being accessible, anywhere. Which wouldn't be the case had it been a protocol.

Post reply on HN