> EVERYTHING MUST FAIL NORMAL.
I'm often surprised how little discussion I see of failure analysis and especially failure design is in software engineering. It is one of the most important concepts in engineering, but I guess we still operate like software can't kill you.
For those unfamiliar with this type of analysis I'll give you 3 examples. 1) Imagine you are building skyscrapers in city with a bunch of other skyscrapers. You don't expect them to fall but you question what happens if they do. You will explicitly design them to fall in on themselves (say after... idk... a plane hits them?) so that they do not fall on their side and take out other buildings around them. An important aspect of security design is explicitly around this whole idea. When your systems fail do you want them to "fail open" or "fail closed"? 2) Fail open would be like TSA, where if you start firing off rounds in the queue the system fails and you get "access" to the rest of the airport. 3) Failing closed is typical of safes, where you accept the fact that you will need to take a blowtorch and drill into the safe. (the TSA thing did happen, just not in America)
While these examples are a bit extreme the ideas are incredibly important for many products. I'll give a good example of this by looking at Android Auto. If a user gets frustrated with the system and is not able to accomplish their task in the way __they want to__, they will disconnect their phone, look down at it, do the thing, and reconnect. No matter how much you want to push voice operations, if the person doesn't like that, they fail over to a more dangerous set of actions than if you had just let them use the console a little longer or let them do a specific task. At least their eyes would be closer towards the road. I'm often amazed that there are lack of consideration for these type of things in home and auto platforms where there are often high stakes. How things fail is one of the most important things when designing systems (I'd argue it even matters for things like law and medicine).