Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Is Critical Life or Death Software Tested?

motherboard.vice.com

Post reply on HN
Print view View original

131–140 of 161 posts

Re: How Is Critical Life or Death Software Tested?

#131

Post by [deleted] »

peteretep wrote: post #123
z3t4 wrote: post #21

Earlier quoted context omitted.

A lot of ppl get this backwards, that a stable program should never "crash". While it's actually the opposite, it should throw errors at every opportunity to do so. The errors should then be logged and the program should be restarted by a watcher process. Here's an example on how you can both log errors and e-mail them if a process crash, using a startup script (Linux, Ubuntu): exec sudo -u user /bin_location /progra…

If an error is caught and handled, calling it a crash seems disingenuous.

[deleted]

Re: How Is Critical Life or Death Software Tested?

#132

Post by acomjean »

noir_lord wrote: post #68
fredmorcos wrote: post #62

Earlier quoted context omitted.

> Reliability was favored over other things (for example recursion was discouraged). This sounds really strange to me. So may I ask why? I find that recursion - most of the time - helps shorten and clarify the code. Also, doesn't recursion make induction proofs trivial?

The issue is largely that unbounded recursion is quite easy to do accidentally (in many of the languages that where used in the past) with the resulting stack smashing causing issues. Also many of these systems where hard real time as in "if we don't respond in under 30ms something expensive goes bang" and again recursion can cause problems with that, lots of these systems are interrupt driven and have no garbage col…

Yup. We had some processes running with their own cpu with interrupts turned off, so if the process went weird it meant reboot (as we discovered the hard way one day). So we tried to keep code simple.

On the plus side you had a pretty good idea about how long the max processing would take (and avoid the timeouts and aforementioned "bang"), as the OS couldn't interrupt us. Certain system calls couldn't be made while in what we called "soft real time". Memory allocation was done upfront.

The process control of that system was interesting. You could assign processes to processors or groups of processors and then give those groups a scheduling method. I haven't seen anything like it in the years since I left.

Re: How Is Critical Life or Death Software Tested?

#133

Post by _nedR »

mlonkibjuyhv wrote: post #61
acomjean wrote: post #26

Earlier quoted context omitted.

Even while running the radar would keep track of communications between the parts and make sure things were still ok. The system needed messages periodically from the external components and vice versa to make sure things were ok. There were status messages sent around too. And a display of how things were doing. Its been a while, but I remember some of the things. You could command the external things to run diagnos…

Cars often disable any system capable of interfering with the wheels at the least sign of issue. Insufficient seal on the fuel-cap? Disable ABS, TCS, etc.

What really scares me are the ABS and traction control systems that are now becoming mainstream on motorcycles. These systems have accelerometers and gyros, which take in to account, information like lean angle of the bike when calculating the braking force. It certainly is a feat of engineering.

Little about being a biker, programmer, and an aspiring tree-shade mechanic, reassures me about the safety of these systems. Internals of a bike are much more exposed to abuse, bikers are known to take a spanner to their machines, the motorcycle repair workshops are a much more informal industry (at least here in India). What happens to a bike that is not subject to regular maintenance? On the other hand, I have always marvelled at how mechanical systems like motorcycles are usually built with some sort of graceful failure in mind - In a lot of cases a motorcycle will warn you about a faulty component before it fails catastrophically. I assume the people who designed these systems, would have kept that in mind while designing these systems (although stuff like Toyota's unintended acceleration does not inspire confidence).

And what happens in the event of a catastrophic failure:- a car locking up at speed is still dangerous, but there is room for error. If the front tires of your motorcycle locks up at speed, the odds of you walking away from the incident are not high.

Don't get me wrong, these systems DO SAVE more LIVES than they could possibly take away in the long run, But I am still disconcerted by the whole thing.

EDIT: typos

Re: How Is Critical Life or Death Software Tested?

#134

Post by learc83 »

Sophistifunk wrote: post #94
lambdaelite wrote: post #59

Earlier quoted context omitted.

A familiar interlock would be the mechanism used to disable a microwave magnetron when the door is in the open position.

Which apparently doesn't work so well, according to radio astronomers :-/ Kinda makes me worry about how many times I've microwaved the meat-n-two-veg just a bit.

Damage to the testicles is caused by the thermal effects of the microwave, so it's likely you would have noticed.

Re: How Is Critical Life or Death Software Tested?

#135

Post by imrehg »

Okay, the LightSail spacecraft is not a "life or death" thing, but, as it was in the news yesterday[1] seriously logging into CSV file onboard can crash the system and have to wait until it reboots itself (don't react to soft reboot either)?

Feels sad, that a lot of lessons learned are getting lost along the way...

[1]: http://www.planetary.org/blogs/jason-davis/2015/20150526-sof...

Re: How Is Critical Life or Death Software Tested?

#136

Post by pdkl95 »

manarth wrote: post #95
pdkl95 wrote: post #87

Earlier quoted context omitted.

Really, every engineer should read the report[1] from the Therac-25 investigation. I would hope anybody that is working on anything that could be potentially dangerous has already read it. There problems in the Therac-25 went a lot further than just the bad design of the target-selection, which had an (badly designed) interlock. It checked that the rotating beam target was in the correct position to match the high/lo…

"The really telling thing, though, is how the previous model acted…it would blow fuses regularly" Good. When a fuse blows, it shows something is wrong, and needs fixing. Replacing the fuse with a nail or something else that doesn't blow is a sure-fire way to set the thing on fire. Bad enough for a desk-lamp, a little worse for radiotherapy machine. Sounds like people were irritated by fuses blowing, and decided to si…

The people using the machine a the hospital would replace the (expensive) fuses when they blew. It was the manufacturer that made the later model (the Therac-25) that didn't have the fuses (and other "old" hardware features).

Obviously, something was still very wrong. User error (or other bugs? I'm not sure) in the older hardware and the infamous race condition in the software-controlled Therac-25 was causing the beam to turn on some shockingly high amount of power. The better design of the older models saved people's lives by simply blowing fuses when the power went too high.

You could, perhaps, blame the poor communication between the hospitals and the manufacturer, because the fuse problem should have cause a bit of a panic among the engineer who designed the machine.

Re: How Is Critical Life or Death Software Tested?

#137

Post by z3t4 »

peteretep wrote: post #123
z3t4 wrote: post #21

Earlier quoted context omitted.

A lot of ppl get this backwards, that a stable program should never "crash". While it's actually the opposite, it should throw errors at every opportunity to do so. The errors should then be logged and the program should be restarted by a watcher process. Here's an example on how you can both log errors and e-mail them if a process crash, using a startup script (Linux, Ubuntu): exec sudo -u user /bin_location /progra…

If an error is caught and handled, calling it a crash seems disingenuous.

One mistake that ppl do is they wrap their code around a try ... catch, where it's better to throw an error and exit. If there's an error in one place, chances are there are also errors elsewhere, so it's better to restart the program instead of continue with a bad state.

When the error gets thrown in your face, there's a higher chance that it gets fixed.

But this also have its setbacks. Loosing the whole state can be really bad.

Re: How Is Critical Life or Death Software Tested?

#138

Post by mrsteveman1 »

jarrettch wrote: post #109

I've been wondering about this lately. My stepdad has heart failure and they put a heart pump in his chest. It regulates blood flow and settings can be changed, etc. Leave it to a dev to think "How much testing has gone into this thing?" Even one minor slip-up in his blood flow, either too high or too low, could mean a stroke and possibly death. Or, God forbid, the thing crashes somehow and stops working. That machin…

I don't know much about heart pumps, but I do have an implanted pacer/defibrillator, and I'm currently not too happy with the people who made it.

I've been told by my cardiologist (and engineers working for the manufacturer) that it doesn't "fail safe" if the battery level drops too low to keep the device running (which is inevitable if it isn't replaced after 7-8 years, but it can and sometimes does happen prematurely and without warning).

In that situation, not only can it suddenly become unable to correct an arrhythmia (as expected), it could actually cause one all by itself, or pace above 200BPM for no reason.

No one I've talked to in the healthcare industry seems at all surprised about this for some reason. They just started monitoring it more often the closer it got to the "replace me now" indicator level.

Re: How Is Critical Life or Death Software Tested?

#139

Post by manarth »

TheLoneWolfling wrote: post #121
manarth wrote: post #86

Earlier quoted context omitted.

"ship both a and b, whose output always has to match exactly or it reports a fault and the component requires replacement." And when that software disagrees…but the plane happens to be at 40,000 feet? The plane just stops running until the component is replaced? I don't know too much about plane hardware, but I scuba-dive a rebreather which has critical life-support electronics. It has two independent computers, and…

Are both computers running the same software? Because if so, that's a single point of failure right there. (Just like multiply redundant hardware won't help you if there's a design flaw and they all fail at once.)

Yes, they are…and that's an acknowledged failure risk. A number of divers mitigate that by having a separate independent fourth O2 cell, monitored by a separate computer (from another manufacturer). Divers are trained to watch for any sign that the computer isn't doing its job (and there are certain typical indicators, although they don't cover every possible scenario), and if there's any doubt, switch to an alternative (either bailout to open-circuit, or switch to a fully-manual mode of operation) - either way, the computer is then out of the picture. Of course, fully-manual mode just isn't possible for, say, a passenger jet.

Re: How Is Critical Life or Death Software Tested?

#140

Post by golergka »

acomjean wrote: post #8

I wrote software for Radars. Kind of important (not like plane software). We used Ada alot, which in my estimation helped. Software was reviewed. Tests were reviewed. Reliability was favored over other things (for example recursion was discouraged). We used Ada's constrained types (this value is between 1 and 99, if it goes out of range, throw an exception). For external hardware inputs, we had software simulating th…

> An when they built the physical radar, the software worked.

I can only imagine a feeling of satisfaction you and your colleagues felt at this moment.

Post reply on HN