Another engineering best practice is to keep proper logs. The Toyota recalls from 2009 to 2011 were likely caused by some bugs but they weren't able to find the root cause. Ostensibly because not enough data were being logged.
How to prevent software bug from killing lives like Boeing's or Tesla's?
11–20 of 20 posts
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#12Start with researching formal "quality management systems." At the very least that'll introduce you to using FMEAs, external standards, rigorous design reviews and testing, quality gates between development and production (if they're not a PITA, they're not working) and traceability for everything in production.
There are off-the-shelf learning materials for all of it.
If you do go down that road hire someone with QMS experience to design your process and handhold your team through the transition. Otherwise you're likely to over-complicate it for a less effective result.
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#13Overall, I think designing a system that prevents people from being harmed is very hard, and has to be a design level concept that is in everyone's minds as they start, and every day through development. Even your example "the worst can happen when our software has problem is someone won't get a meal on time or a meal is wasted." isn't really true. What if you ordered fish, or oysters, and they were left out too long…
You were right with the counter-exaple that we have redundancy in place that is 'hardware' - mealbox stickers, production qa.
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#14Follow engineering best practices. One thing you should never do is to write a new system, requiring it to perfectly emulate an old one. It can never work and there will always be unexpected deviations in system behavior. I.e. in Boeing's case it isn't so much about any particular flaws (faulty AoA sensor etc), it is about the whole idea of having a completely new aircraft design appearing to the pilot as if it was a…
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#15Redundancy.
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#16Follow engineering best practices. One thing you should never do is to write a new system, requiring it to perfectly emulate an old one. It can never work and there will always be unexpected deviations in system behavior. I.e. in Boeing's case it isn't so much about any particular flaws (faulty AoA sensor etc), it is about the whole idea of having a completely new aircraft design appearing to the pilot as if it was a…
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#17If you mean software development for mission critical things that control movement like aircraft, drones, factory robots etc., these engineers I would assume use verified compilers/toolchains like CompCert project to implement the models they have already formally analyzed http://symbolaris.com/course/fcps17.html but I've never done mission critical work, just dabbled in it to apply methods there to non-critical software.
[1] https://arxiv.org/pdf/1611.02273.pdf - Application-layer Fault-Tolerance Protocols
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#18Anyway, there are a few things anyone can do against human stupidity.
The most important thing is to have a decent backbone as an engineer. Do not take shortcuts in safety even if your stupid or arrogant boss tells you to do so.
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#19The failure on the MAX was related to computers moving controlled surfaces (flaps, rudder, etc.) on the plane automatically based on sensor readings. Questions should be asked about what problem that was solving, and was that a requirement (aka complaint) from pilots? Why was that feature necessary?
Re: How to prevent software bug from killing lives like Boeing's or Tesla's?
#20A simple approach would be to not automate things that don't need automating. I think as developers we get a little too far ahead of ourselves. The failure on the MAX was related to computers moving controlled surfaces (flaps, rudder, etc.) on the plane automatically based on sensor readings. Questions should be asked about what problem that was solving, and was that a requirement (aka complaint) from pilots? Why was…