Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Complex Systems Fail (1998)

how.complexsystems.fail

Post reply on HN
Print view View original

61–70 of 90 posts

Re: How Complex Systems Fail (1998)

#61

Post by tptacek »

pdonis wrote: post #34
tptacek wrote: post #16

I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing. The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure…

> "root cause analysis", at least on complex systems, is a fools errand. Not when properly understood--which, to be fair, it often isn't. The proper purpose of root cause analysis is to answer the question: does this failure mean I need to change something about this system? If the answer turns out to be yes, the thing that you need to change gets called the "root cause" of the failure. Of course that's a misnomer, s…

Cook doesn't object to postmortems and to using hard failures as opportunities to spot interventions (though, remember, all practitioner actions are gambles). The issue is in attempting to identify a single privileged root cause.

Re: How Complex Systems Fail (1998)

#62

Post by tptacek »

esafak wrote: post #51
cpgxiii wrote: post #49

Earlier quoted context omitted.

The "naive notion of system performance" is that the system is primarily operating in the ideal regime, and thus all deviations deserve alerts (and receive attention). The reality is that often the system is operating often enough in some failure mode that any such alerts would be regularly triggered by accepted behavior, and thus the alerts are systematically ignored/disabled/normalized. E.g. nominally you should ne…

My response would be to use more sophisticated anomaly detection.

Problems managing a complex system? Why not consider adding another complex system!

Re: How Complex Systems Fail (1998)

#63

Post by tptacek »

ErroneousBosh wrote: post #41
tptacek wrote: post #27

Earlier quoted context omitted.

I am also a broken record about Grug-brainedness, but I don't need to be so repetitive about it because so many other people are too.

My Grug-brained hill to die on is that there are many pieces of technology like the LM741 opamp, the Rover V8, Morris A-series and Peugeot XUD engines, and a big SQL database running on a thumping great Unix box, that have remained essentially unchanged for decades, worked well when they were new, and are still a perfectly good solution today for so much of what you want to do. You know what they have in common? A la…

All things equal you're always better off ruthlessly killing complexity. Cook (I know you're not rebutting him) is focused on the problem of unavoidably complex systems. A big server running a single instance of Postgres is great, and you should absolutely use that architecture if you can. But real-world constraints can break the architecture: need it running in multiple locations, with a single global view, with strictly bounded propagation times (for instance: because you're running a routing protocol): it's still just a database problem, but now it's a much more complex one.

And then, naturally, you can get complexity just by building stuff on top of that big database server, and other things that depend on those things.

Re: How Complex Systems Fail (1998)

#64

Post by otterley »

jonahx wrote: post #44
jasode wrote: post #38

Earlier quoted context omitted.

>The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. It's only an issue when people take that phrase very literally. People have common sense to understand that things have multiple causes and a chain of events. NASA has "Root Cause Analysis" (singular) all over various official documentation and it doesn't stop them from understanding that th…

> Another cause was the unrealistic flight schedules which can subconsciously pressure management into normalizing dangerous deviations. In the case of Challenger, I think it's pretty clear this was the "root cause": > According to testimony by Kilminster and Boisjoly, Mason finally turned to Bob Lund and said, "Take off your engineering hat and put on your management hat." Joe Kilminster wrote out the new recommenda…

If you’re doing “five* whys” analyses correctly, you don’t stop at the technical causes. You continue further to analyze the causes that precipitated the technical errors, too. This exposes the business reasons behind them and forces management to face them.

* the number five isn’t magical here. You don’t have to stop at five, and you often shouldn’t.

Re: How Complex Systems Fail (1998)

#65

Post by AlotOfReading »

YZF wrote: post #25
tptacek wrote: post #16

I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing. The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure…

Root Cause Analysis is an organizational ritual that makes people feel like they're doing something. It's more about politics than about engineering. Not only is the process not useful- more often than not the correct root cause isn't even correctly identified. Basically a poorly designed system is going to fail somewhere. I like to think about it either as the part of the iceberg that's visible or my other analogy i…

It's notable how many more engineers are paying attention to the details in mechanical systems. Take a random electrical connector in your car. Likely dozens of engineers were involved in its design and manufacturing. Dozens were involved with incorporating it into the vehicle design and installing it during manufacturing. There are entire teams who specialize in e.g. high-vacuum flange seal design. It's pretty rare to find similar teams in software.

I will argue that mechanical systems are fundamentally simpler than software, though. Try writing a datasheet for some library as an exercise. Thoroughly documenting the interface is tedious, but straightforward. You need graphs though, so maybe performance under load? But that's assuming a lot about hardware and runtimes, so you need to characterize it under all of those possible degrees of freedom. That's a much larger space than physical components have. Imagine characterizing/qualifying parts under disparate manufacturing methods that include "some sleep-deprived rando who's never heard of a lathe needs to build this successfully in his garage" (i.e. the norm in OSS).

And even if you successfully write a datasheet, some unrelated component elsewhere in the system can remotely interfere with your software through chained vulnerabilities, memory pressure, contention, etc. Mechanical engineers rarely worry about entirely disconnected parts interfering in undocumented ways, because reality limits how things can interact. RF engineers consider this kind of problem more regularly, and need correspondingly more animal sacrifices. The larger the software the worse this problem becomes, and small software systems are increasingly endangered these days.

Re: How Complex Systems Fail (1998)

#66

Post by robwwilliams »

_boffin_ wrote: post #39
tptacek wrote: post #16

I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing. The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure…

Check out the book called, The Logic of Failure. I think you’ll like it

Two other great books on failures in inevitably complex military operations are:

1. Command in War (1985) by Martin van Creveldt. He dissects may complex but common military failure causes up to the Vietnam War.

2. Team of Teams (2015) by Stanley McChrystal that is focused on what he and teams learned in Iraq but extrapolating to most organizations.

Re: How Complex Systems Fail (1998)

#67

Post by rglover »

feyman_r wrote: post #2

I may have shared this before on a different submission: John Gall’s books are really good on this topic: General Systemantics [ https://en.wikipedia.org/wiki/Systemantics ]

Love Systemantics. That was one of the first systems books I read and where I felt like I found someone practiced who understood how systems actually function (a realistic, honest view that systems always fail, it's just a question of when, why, and how).

Re: How Complex Systems Fail (1998)

#68

Post by DrewADesign »

marcosdumay wrote: post #48
DrewADesign wrote: post #35

Earlier quoted context omitted.

I don’t blame people for being confused about the term in software — it’s a bit more straightforward in manufacturing, for example. That said, if you look at a system that failed and determine a root cause analysis is pointless, you may have just stumbled upon your root cause.

> it’s a bit more straightforward in manufacturing That's because manufacturing is a simpler environment where causes and consequences are more conceptually distinct. Until you start to pull management decisions into your analysis, of course. Then it becomes just as complex as software.

There’s even more variety in industrial flows than software architectures, and some of them are pretty fucking complex, but the steps are usually better defined, the processes more deliberate, and the custody of material and process oversight is better documented… at least for manufacturers in more advanced countries that are far more liable for product defects. You can dick around with software or software services with relative impunity, but if you release a product with a dangerous manufacturing defect, or lose a few runs of expensive material, the bottom line is not as forgiving. I’m sure there are places in developing countries burning waste plastic to make garbage pot metal products or something for which that level of oversight provides near zero ROI, but outside of that, it’s usually just a lot more deliberate.

Re: How Complex Systems Fail (1998)

#69

Post by _boffin_ »

robwwilliams wrote: post #66
_boffin_ wrote: post #39

Earlier quoted context omitted.

Check out the book called, The Logic of Failure. I think you’ll like it

Two other great books on failures in inevitably complex military operations are: 1. Command in War (1985) by Martin van Creveldt. He dissects may complex but common military failure causes up to the Vietnam War. 2. Team of Teams (2015) by Stanley McChrystal that is focused on what he and teams learned in Iraq but extrapolating to most organizations.

Did love Team of Teams. Need to check out the other

Re: How Complex Systems Fail (1998)

#70

Post by elisbce »

"Catastrophe requires multiple failures – single point failures are not enough."

I disagree with this point. From what I have seen and experienced, there are so many SPOFs in the real-world complex systems that do not have adequate defense or even any defense at all. The only reason catastrophe didn't occur (often) is only because the natural failure rate is extremely low. Like the probability of air traffic controller making a critical mistake is very low, maybe once every few years, but when it happens, the SPOF will almost certainly cause a disaster, and the system (ie. the way air traffic is controlled today) has been around for decades and is so outdated but the cost of changing the entire process around the world is simply way too high so we just take this as "normal".

Post reply on HN