81–90 of 161 posts
If Amazon can do it, then surely an airplane manufacturer can do it [1] [1] http://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-s...
I wrote software for Radars. Kind of important (not like plane software). We used Ada alot, which in my estimation helped. Software was reviewed. Tests were reviewed. Reliability was favored over other things (for example recursion was discouraged). We used Ada's constrained types (this value is between 1 and 99, if it goes out of range, throw an exception). For external hardware inputs, we had software simulating th…
> Reliability was favored over other things (for example recursion was discouraged). This sounds really strange to me. So may I ask why? I find that recursion - most of the time - helps shorten and clarify the code. Also, doesn't recursion make induction proofs trivial?
If you are careful about test vectors, you can pseudo-exhaustively prove that an iterative dual to a recursive algorithm is equivalent.
It's all just a question of cost. We know how to write software that comes arbitrary close to perfection. But as defects asymptotically approach zero, cost skyrockets. The interesting question is what technologies can bend that cost/quality curve.
This is why this discussion sometimes frustrates me. A lot of the defects we have are because you aren't willing to pay for the sort of software that wouldn't have defects. It's natural to read that as a sort of cynical accusation, but instead, I mean it straight... you really aren't willing to pay what it would take, and you shouldn't be. A $1000 Facebook-access app for your phone (that still somehow has some sort o…
Earlier quoted context omitted.
> Actually testing was built into the software. When it came up it would talk to the physical parts to make sure everything was communicating ok before it could start running. I've been wondering about this for a while. We tend to run unit tests, integration tests, whatever tests, while the software is in development. However, once it is in "production" (for whatever definition of production), usually no tests are pe…
Early in my career I worked on military flight data recorders, including the development of the software for the F-22's "black box". Those systems have SBIT, IBIT, PBIT and MBIT sub-systems were BIT is "built in test" and S = startup, I = initiated, P = periodic and M = maintenance. I remember making the Star Trek diagnostic joke myself when I was assigned the SBIT work. Each BIT does varying level of testing based o…
I think the article might be mistaken about one point: "For one thing, the Boeing approach is going out of style or has mostly gone out of style, according to SE poster Uri Dekel (handle: Uri), a Google software engineer." It absolutely has not gone out of style in avionics software engineering. As a person who writes software for avionics, I can say that extensive design reviews at every step combined with rigorous…
[deleted]
And when that software disagrees…but the plane happens to be at 40,000 feet? The plane just stops running until the component is replaced?
I don't know too much about plane hardware, but I scuba-dive a rebreather which has critical life-support electronics.
It has two independent computers, and three O2 oxygen-pressure cells. The 3 cells report their reading of O2 pressure to both computers. Both computers simultaneously display the pressure on independent displays. One computer is primary (active, controlling the O2 pressure), whilst the secondary is display-only.
Both computers use a majority-rule…the two oxygen cells with the closest value win, whilst the third is ignored. This could potentially be fatal - two failing cells can report incorrect pressures and win the vote - so rebreather divers are also taught manual techniques to validate the computer readings (such as a diluent flush which is expected to produce a known predictable reading).
So there are a few techniques beyond simple a/b testing: best of 3 (or more, if available); independent circuits (ideally designed + built by independent manufacturers); manual techniques to give human verification of the data.
Whilst I certainly don't have extensive knowledge of real-time/safety-critical systems, it's clear that there are a lot of techniques, processes and procedures that we wouldn't necessarily be aware of in unrelated tech (e.g. web-dev) that do directly relate to that subject, and might well solve many of the scenarios we come up with.
Earlier quoted context omitted.
Not sure what you mean by "interlocks", but the hardware was quite distributed. Each critical component had its own board and industrial microcontroller. And we had various levels of watchdogs keeping track of system health at all times.
Interlocks are usually fairly crude safety measures, normally in hardware, to make sure that particular combinations of events cannot happen. The Therac-25 is a famous comp.risks cautionary tale. Among the many, many design misfeatures (if you haven't come across it, it's worth a read) was the one that killed people: It was capable of providing two kinds of radiation therapy; electron beam radiation and X-ray radiati…
There problems in the Therac-25 went a lot further than just the bad design of the target-selection, which had an (badly designed) interlock. It checked that the rotating beam target was in the correct position to match the high/low power setting (and NOT the 3rd "light window" position without any beam attenuator).
While many design choices contributed to the machine's problems, you could probably say that two big design failures lead to the deaths associated with Therac-25. One was this interlock, which failed if you didn't put it in place (there was no locking mechanism, either, just a friction stopper). If the target was turned slightly, the 3 micro-switches would sense the wrong pattern (bit shift)... which was pattern for one of the OTHER positions.
There was also a race condition in the software that would turn on the beam at a power MUCH higher than it is ever used. This race was only triggered when you typed in the treatment settings very quickly, which is why the manufacturer denied there was a problem: when they tried to recreate the bug by carefully - that is, very slowly - following the reported conditions, it never failed.
Therac-25 is an incredibly powerful lesson in what we mean by "Fail Safe", and why it is absolutely necessary to have defense in depth. Fixing the target wouldn't have fixed the race condition power-level bug. Fixing any of the software wouldn't have fixed the bad target design that could be turned out of alignment. Oh, and they had a radiation sensor on the target (which could shut off the machine as another independent layer of defense... but they mounted it on the turnable target, so the micro-switch problem allowed the sensor to be moved away from the beam path.
The really telling thing, though, is how the previous model acted. It was not software controlled, and was an old-style electromechanical device. It turns out the micro-switch problem existed there as well (among other problems)... and it would blow fuses regularly. Which was yet another layer of safety. It turns out that when they upgraded it to a software-based control system, they got cheap and took out all those "unnecessary" hardware interlocks and "redundant" features. There is a lot of blame to go around, but this is where I put most of the responsibility. You never assume one (or even a few) safety feature will work - the good engineer assumes it will all break at any moment, and makes sure that it will still Fail Safe.
> (although it's possible for interlocks to go wrong too)
If there is one lesson to learn from the Therac-25, this was it. Things break, mistakes happen, and when you're building a device that shoots high-energy x-rays at people, you need to assume that everything did go wrong, and make sure the rest of the device can safely handle that situation.
It's all just a question of cost. We know how to write software that comes arbitrary close to perfection. But as defects asymptotically approach zero, cost skyrockets. The interesting question is what technologies can bend that cost/quality curve.
Not technologies, process. I am 90% sure that actually taking any organisation and committing to good known process will raise the game by orders of magnitude - so technologies supporting and enforcing said process will be of benefit And I think the process looks like this 1. Written requirements up front 2. Total isolation / integration points defined and contractually enforced 3. Test harnesses built first 4. Per r…
#2 and #3 might be feasible, but they will require a massive shift of perception across most stakeholders. It's a politics game, and you will need the perennial support of a very influential sponsor to push through the feet dragging phase.
#4 Might be easier to sell, but still require time and effort to implement. In a sense, this is ultimately also a political matter.
If Amazon can do it, then surely an airplane manufacturer can do it [1] [1] http://cacm.acm.org/magazines/2015/4/184701-how-amazon-web-s...
that Amazon paper is talking about design/algorithm verification using formal methods. This does NOT verify that the design or algorithm in question is actually implemented correctly and matches the design perfectly.
[deleted]
"ship both a and b, whose output always has to match exactly or it reports a fault and the component requires replacement." And when that software disagrees…but the plane happens to be at 40,000 feet? The plane just stops running until the component is replaced? I don't know too much about plane hardware, but I scuba-dive a rebreather which has critical life-support electronics. It has two independent computers, and…