Live data from Hacker News

How Complex Systems Fail (1998)

how.complexsystems.fail

21–30 of 90 posts

Re: How Complex Systems Fail (1998)

#21
Overall a good collection on the complex systems, somehow it missed the part how complex system came into the existence in the first place itself.

> Human expertise in complex systems is constantly changing

I feel this is single most important factor responsible to both making system complex and at the same time improving them depending on who the people are and how they take failures and breakdowns. I find it funny but we are in an era where folks building ML systems do not seem to remember the direction in which to open the screws. They do have clear expertise in something new but clearly lack in some other areas

Re: How Complex Systems Fail (1998)

#22
post #6

> Failure free operations require experience with failure. This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.

I've always struggled to apply this to the systems I work on. If the system fails, someone potentially dies, though in practice they've never been more than hospitalized. To avoid that, huge amounts of effort are expended on failure modelling and testing, but that doesn't eliminate unknown unknowns. That discrepancy has made front page news a couple times.

When I was at Netflix and reddit, one thing I said often was, "Luckily, we are not a bank". And it sounds like you are working on even more critical systems than that.

Chaos engineering doesn't really apply to data critical or safety critical systems. You can't just break them in the real world to see how they fail.

You have it exactly right -- it has to be modeled and tested in lab conditions. Safety critical systems are not a place for YOLO development.

Re: How Complex Systems Fail (1998)

#23
post #6

> Failure free operations require experience with failure. This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.

It's much more universal and complicated than that. One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Attempts to solve the issue look often like "Hey, kids, let's fall over now all at once in safest way possible and learn from it!". But it doesn't work at all. Real failures have to be un…

> One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures.

Its cause we already learned from our failures and make the world a safer place. In my youth we climbed on trees, one of my friend fall down broke his arm, the doctor couldn't fix it and he can not move the hand for the rest of his live. My friend would be happy do not been allowed to climb, or at least under safer conditions but have his hand moveable still. I burned myself really heavy with fireworks as teenager and have a huge scar from it. I never touched any firework after this but also never bought it for my kids, so they never had the chance to "learn" from making the the same mistake. Keep in mind that while claiming the world is too safe you say this from a perspective of an survivor.

Re: How Complex Systems Fail (1998)

#24
post #6

> Failure free operations require experience with failure. This is why we created Chaos Engineering. By constantly forcing failure, it made us always create systems in defense of that failure, and gave us great data on where the tipping point is for different systems within a particular failure mode.

It's much more universal and complicated than that. One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Attempts to solve the issue look often like "Hey, kids, let's fall over now all at once in safest way possible and learn from it!". But it doesn't work at all. Real failures have to be un…

I agree with you. Kids aren't allowed to fail anymore. And as a parent, when I try to let my kid fail, I get scolded for being a bad parent. Broken bones used to be a right of passage for kids. Now it happens far less. While a broken bone sucks, it teaches you important lessons! Don't do that again, and hey, you messed up but you're fine now.

Re: How Complex Systems Fail (1998)

#25
post #16

I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing. The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure…

Root Cause Analysis is an organizational ritual that makes people feel like they're doing something. It's more about politics than about engineering. Not only is the process not useful- more often than not the correct root cause isn't even correctly identified.

Basically a poorly designed system is going to fail somewhere. I like to think about it either as the part of the iceberg that's visible or my other analogy is lighting doesn't strike twice at the same place. A robust/well designed system is just very different than a brittle/poorly designed system. The brittle system will exhibit random failures and trying to chase them is counterproductive.

We do have systems that are fairly complex and reliable. Take an internal combustion engine as one example or an entire car or an airplane (well, one of the well designed ones). Then we build bigger systems on top of that, a city's transportation system, where we do get to scales where things "fail" all the time. Yes, some bus somewhere or some train somewhere doesn't get to a station at the correct time, because a bus breaks down or heavy traffic or whatnot. But still a well designed system is robust to those. The parts we build the systems out of are well understood and so the resulting reliability can also be understood.

With software specifically we tend to not design and build that well. We throw stuff together. Then we're surprised when they fail. And we make excuses by saying "complex systems". I often feel it's our lack of discipline and skill vs. other domains and not really that our systems are that much more complex.

EDIT: Some might argue that mechanical systems like engines are just fundamentally simpler than software. But look at something like a bearing. Just one bearing is an incredibly complex system. Bearing failure is a statistical event, some bearings fail sooner, some fail later. There are centuries of know-how in the bearings used in said engine. Mechanical engineers generally use well understood components/designs and build in margin to meet the requirements of the engine they're designing. In software we often re-invent the "bearing" or we'll pick some new unproven design for a "bearing" and we'll generally build without a good understanding of the performance or failure modes of our "bearings" and without a known pattern as to how to use them and when to use them. And then we're surprised when our "car" fails in weird ways. We also don't test to the same standard that a mechanical engineer would use for a new design. They would have a room full of widgets doing ten million cycles before they accept the design as part of a larger system. We "yolo" it...

Re: How Complex Systems Fail (1998)

#26

Earlier quoted context omitted.

It's much more universal and complicated than that. One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Attempts to solve the issue look often like "Hey, kids, let's fall over now all at once in safest way possible and learn from it!". But it doesn't work at all. Real failures have to be un…

> One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Its cause we already learned from our failures and make the world a safer place. In my youth we climbed on trees, one of my friend fall down broke his arm, the doctor couldn't fix it and he can not move the hand for the rest of his live.…

There's a fine line between "we made the world safer" and "we made the world too constricting to learn". I would like my kids to be allowed to fail a bit more often, while still staying safe. I think we've swung too far in the safe direction.

Re: How Complex Systems Fail (1998)

#27
post #16

I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing. The most commonly cited subtext or thrust of it is that "root cause analysis", at least on complex systems, is a fools errand. Something goes wrong, say, in a distributed lock system, and your whole deployment system enters a metastable failure…

> I'm a broken record on how important I think this document is, and that it's hard to appreciate it until you've had extended experience with complex systems actually failing. If you want a bit of cheese to go with that wine, this article pairs nicely with The Grug-Brained Developer: https://grugbrain.dev/

I am also a broken record about Grug-brainedness, but I don't need to be so repetitive about it because so many other people are too.

Re: How Complex Systems Fail (1998)

#30
post #24

Earlier quoted context omitted.

It's much more universal and complicated than that. One of the big issues schools and pedagogy in general struggle with is that our environment is far too safe in too many ways. For kids there is too few ways to learn from failures. Attempts to solve the issue look often like "Hey, kids, let's fall over now all at once in safest way possible and learn from it!". But it doesn't work at all. Real failures have to be un…

I agree with you. Kids aren't allowed to fail anymore. And as a parent, when I try to let my kid fail, I get scolded for being a bad parent. Broken bones used to be a right of passage for kids. Now it happens far less. While a broken bone sucks, it teaches you important lessons! Don't do that again, and hey, you messed up but you're fine now.

I feel like painfully falling without breaking bones are pretty good lessons already, and I don't quite see what the broken bones add there.
Post reply on HN