Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How complex systems fail (2002) [pdf]

web.mit.edu

Post reply on HN
Print view View original

11–20 of 22 posts

Re: How complex systems fail (2002) [pdf]

#11

Post by adrianratnapala »

rossdavidh wrote: post #8

This all looks spot on from my experience, except I have this nagging voice in my head that says the methods they are criticizing, seem to be exactly what the airline industry has done for decades in the wake of plane accidents. The result has been an ever-improving safety record, without preventing normal operation; although it is a pain to get through security at the airport, people do in fact continue to get from…

I know nothing of the industry, so everything that follows is guesswork.

I'd say airlines are a sweet spot because we are probably near a technological local-optimum. We are not constantly evolving new paradigms for the industry, rather just incrementally improving the old one.

If post-mortem culture in the industry results in "fighting the last war", then that probably does little harm, at worst it adds in incremental cost. In the mean-time there is still incremental techonolgical improvment giving us more head-room in the cost vs. safety trade-off.

Re: How complex systems fail (2002) [pdf]

#12

Post by jodrellblank »

qualitytime wrote: post #6

TL;DR Spend 80% on expensive consultants qualified in "project management method" such as PRINCE2/V-Model and make a f *g nightmare for the people who actually try to make it work. A picture is worth a thousand words: https://upload.wikimedia.org/wikipedia/commons/8/8f/Systems_...

Did you even read it? There's no mention of PRINCE2 or any recommendations in it at all

Re: How complex systems fail (2002) [pdf]

#13

Post by xenity7 »

qualitytime wrote: post #6

TL;DR Spend 80% on expensive consultants qualified in "project management method" such as PRINCE2/V-Model and make a f *g nightmare for the people who actually try to make it work. A picture is worth a thousand words: https://upload.wikimedia.org/wikipedia/commons/8/8f/Systems_...

It seems you've been hurt recently by a consultant :).

The institutional response to managing the risk of complex systems is often to introduce layers of approval and process that appear to be dealing with a problem ("this system can't fail, therefore please check with everyone involved in the system before making a change") but really have little real world value except driving everyone crazy and incurring enormous cost. They also shield individuals from real responsibility (how carefully do you review something you are the only approver on? What if there are 5 approves? What about 15?).

A better answer is to find ways to conduct real world tests with subsets of your system where you can roll back bad consequences. As far as I can tell This is the approach of google/Facebook and other newer tech companies with pushing small changes to subsets of customers for testing.

Legacy enterprise companies are woefully inequipped to do this for the most part, both technically and culturally.

Some industries have regulations that make this approach difficult (financial services) or human consequences to failure that make the cost of experimentation too high (medicine, airlines, etc)

Re: How complex systems fail (2002) [pdf]

#14

Post by csours »

If this is interesting to you, I highly recommend "The Field Guide to Understanding Human Error" by Sidney Dekker - it covers these points with examples. [0]

Another note, I wondered what the root cause of the financial meltdown was for a number of years, but looking at it from this point of view, it's obvious that a number of things have to go wrong simultaneously; but it is not obvious beforehand which failed elements, broken processes, and bypassed limits lead to catastrophe.

For your own business/life, think about things that you live with that you know are not in a good place. Add one more problem and who knows what gives.

This is not intended to scare or depress, but maybe have some compassion when you hear about someone else's failure.

0 https://www.amazon.com/Field-Guide-Understanding-Human-Error...

Re: How complex systems fail (2002) [pdf]

#15

Post by rossdavidh »

adrianratnapala wrote: post #11
rossdavidh wrote: post #8

This all looks spot on from my experience, except I have this nagging voice in my head that says the methods they are criticizing, seem to be exactly what the airline industry has done for decades in the wake of plane accidents. The result has been an ever-improving safety record, without preventing normal operation; although it is a pain to get through security at the airport, people do in fact continue to get from…

I know nothing of the industry, so everything that follows is guesswork. I'd say airlines are a sweet spot because we are probably near a technological local-optimum. We are not constantly evolving new paradigms for the industry, rather just incrementally improving the old one. If post-mortem culture in the industry results in "fighting the last war", then that probably does little harm, at worst it adds in increment…

Well this makes some sense, since the speed of sound pretty much stopped technological progress toward every-higher speeds a few decades past.

Re: How complex systems fail (2002) [pdf]

#17

Post by jodrellblank »

rossdavidh wrote: post #15
adrianratnapala wrote: post #11

Earlier quoted context omitted.

I know nothing of the industry, so everything that follows is guesswork. I'd say airlines are a sweet spot because we are probably near a technological local-optimum. We are not constantly evolving new paradigms for the industry, rather just incrementally improving the old one. If post-mortem culture in the industry results in "fighting the last war", then that probably does little harm, at worst it adds in increment…

Well this makes some sense, since the speed of sound pretty much stopped technological progress toward every-higher speeds a few decades past.

Really?

The ancient ones, the humans who lived before my birth - those who launched rockets to the moon and detonated city destroying bombs with atomic power, feats no country has matched in these declining years of civilisation - they also had faster than sound passenger aircraft.

Although it might be the tendency to exaggerate stories with time, rose-tinted glasses and the myth of the noble-savage.

Re: How complex systems fail (2002) [pdf]

#18

Post by taneq »

jodrellblank wrote: post #17
rossdavidh wrote: post #15

Earlier quoted context omitted.

Well this makes some sense, since the speed of sound pretty much stopped technological progress toward every-higher speeds a few decades past.

Really? The ancient ones, the humans who lived before my birth - those who launched rockets to the moon and detonated city destroying bombs with atomic power, feats no country has matched in these declining years of civilisation - they also had faster than sound passenger aircraft. Although it might be the tendency to exaggerate stories with time, rose-tinted glasses and the myth of the noble-savage.

There's a common theme amongst many of those heroic feats: They tended to occasionally catch fire, fall out of the sky and/or explode at inopportune times.

It's the difference between prototype and a production. The Saturn V rocket and the Concorde were both prototype-grade vehicles. The marvel of them is that they worked at all. Now that we've proved the point, we won't put either "into production" unless it's reliable enough to be boring.

Re: How complex systems fail (2002) [pdf]

#19

Post by aaight »

qualitytime wrote: post #6

TL;DR Spend 80% on expensive consultants qualified in "project management method" such as PRINCE2/V-Model and make a f *g nightmare for the people who actually try to make it work. A picture is worth a thousand words: https://upload.wikimedia.org/wikipedia/commons/8/8f/Systems_...

"The [Apollo 17] rocket, which of course never could have held to so true a course without computers, leapt into the sky, hell-bent for the cosmos like some naming avenger, a Spacewar nightmare... The sight affected Gosper profoundly. Before that night, Gosper had disdained NASA's human-wave approach toward things. He had been adamant in defending the AI lab's more individualistic form of hacker elegance in programming, and in computing style in general. But now he saw how the Real World, when it got its mind made up, could have an astounding effect. NASA had not applied the Hacker Ethic, yet it had done something the lab, for all its pioneering, never could have done. Gosper realized that the ninth-floor hackers were in some sense deluding themselves, working on machines of relatively little power compared to the computers of the future yet still trying to do it all, change the world right there in the lab. And since the state of computing had not yet developed machines with the power to change the world at large certainly nothing to make your chest rumble as did the NASA operation all that the hackers wound up doing was making Tools to Make Tools. It was embarrassing."

- Steven Levy, Hackers, Heroes of the Computer Revolution

Re: How complex systems fail (2002) [pdf]

#20

Post by di4na »

rossdavidh wrote: post #8

This all looks spot on from my experience, except I have this nagging voice in my head that says the methods they are criticizing, seem to be exactly what the airline industry has done for decades in the wake of plane accidents. The result has been an ever-improving safety record, without preventing normal operation; although it is a pain to get through security at the airport, people do in fact continue to get from…

Except not.

The air safety Human Factor culture have moved away from what they criticise 30 years ago. They are now fully on Safety II.

I advise Steven Shorrock work recently at https://humanisticsystems.com

Post reply on HN