Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Is Critical Life or Death Software Tested?

motherboard.vice.com

Post reply on HN
Print view View original

121–130 of 161 posts

Re: How Is Critical Life or Death Software Tested?

#121

Post by TheLoneWolfling »

manarth wrote: post #86
[deleted] wrote: post #80

[deleted]

"ship both a and b, whose output always has to match exactly or it reports a fault and the component requires replacement." And when that software disagrees…but the plane happens to be at 40,000 feet? The plane just stops running until the component is replaced? I don't know too much about plane hardware, but I scuba-dive a rebreather which has critical life-support electronics. It has two independent computers, and…

Are both computers running the same software? Because if so, that's a single point of failure right there.

(Just like multiply redundant hardware won't help you if there's a design flaw and they all fail at once.)

Re: How Is Critical Life or Death Software Tested?

#122

Post by ufmace »

superuser2 wrote: post #33
eck wrote: post #16

Earlier quoted context omitted.

> statement about the software being bug free I bet they required the guy who delivered the rocket fuel to sign something saying it contained no impurities, the guy who delivered the external tank to sign something saying it did not leak, etc... why should the software guy be special? Yeah I know. We're special. But the world doesn't always see it that way.

No. That it contained under n ppm of impurities, that it did not leak more than 0.001%/year, maybe. Only a Sith (and consumer product marketing that is lying to you) deals in absolutes. In the durable goods world, you don't pretend things are perfect. Failure modes are designed and disclosed, replacement of parts is expected and made reasonable, tolerances are marked, failure rate metrics like MTBF are known, and as…

That reminded me of phone plans, oddly enough. For a while, everybody was advertising their data plans as unlimited, and then the tech press would get all upset when they found the limit.

I always thought the whole thing was dumb - of course there's a limit. Or are we supposed to believe we can push megabytes/s nonstop all month? I'd rather have them just tell me what the limit is and what happens when you go over it than pretend it's unlimited. And stop having the tech press act like the sky is falling when they discover that the unlimited plan actually has a limit.

Re: How Is Critical Life or Death Software Tested?

#123

Post by peteretep »

z3t4 wrote: post #21
acomjean wrote: post #8

I wrote software for Radars. Kind of important (not like plane software). We used Ada alot, which in my estimation helped. Software was reviewed. Tests were reviewed. Reliability was favored over other things (for example recursion was discouraged). We used Ada's constrained types (this value is between 1 and 99, if it goes out of range, throw an exception). For external hardware inputs, we had software simulating th…

A lot of ppl get this backwards, that a stable program should never "crash". While it's actually the opposite, it should throw errors at every opportunity to do so. The errors should then be logged and the program should be restarted by a watcher process. Here's an example on how you can both log errors and e-mail them if a process crash, using a startup script (Linux, Ubuntu): exec sudo -u user /bin_location /progra…

If an error is caught and handled, calling it a crash seems disingenuous.

Re: How Is Critical Life or Death Software Tested?

#124

Post by TheLoneWolfling »

eropple wrote: post #119
TheLoneWolfling wrote: post #117

Earlier quoted context omitted.

It does slightly more than that - it provides a cheap computation of the upper bound of how long it will take.

Yeah, that too. Come to think of it, that was probably more his concern than just halting.

Going out on a limb here, but I don't suppose it was for a superoptimizer?

Re: How Is Critical Life or Death Software Tested?

#125

Post by ISL »

ploxiln wrote: post #60
manarth wrote: post #43

Earlier quoted context omitted.

You just have to look at Therac 25 for the risks of relying on software interlocks alone. One of the prevailing pieces of feedback was the lack of hardware interlocks - whether that's possible on anesthesia machines I don't know…but the prevailing wisdom is to use/include hardware interlocks wherever that's feasible, for any critical life-supporting equipment.

I don't disagree that "hardware interlocks" are a good idea for such equipment. But now that I think about it, I'm annoyed that we can't know if software was competently written and tested - is it really that different from hardware in that respect? How about, we require that the source be open, and if it's too convoluted for the hospital's respected experts to check, then it fails inspection? Hardware uses many stan…

Hardware kills people too, sometimes in subtle ways. http://en.wikipedia.org/wiki/United_Airlines_Flight_232

The combined utility of hardware and software interlocks is that they're complementary:

Sometimes it's easy to specify an interlock in the language of hardware: Never, under any circumstances, should it be possible to slew an avalanche-control howitzer to point at permanent structures; let's use a steel pipe to block the barrel from traversing beyond safe limits.

Sometimes it's easy to specify an interlock in software: Never, under any circumstances, should a rocket launch unless every desk at mission control has authenticated their assent with the main control system.

When interacting with the real world, real-world interlocks are handy, but they're hardly sufficient to guarantee safety. Nothing is.

Re: How Is Critical Life or Death Software Tested?

#126

Post by flogic »

mturmon wrote: post #71
phleet wrote: post #63

Earlier quoted context omitted.

It's a lot harder to reason about memory constraints on recursive programs. The clearness and correctness of the code often ignores the possibility for stack overflow. Most naive implementations of DFS will hit the stack limit given trees that are all one long path from a single root to a single leaf.

Further to your point, here's the guideline, and rationale, from Gerard Holzmann's document on recommended coding practices for C at NASA/JPL: 1. Rule: Restrict all code to very simple control flow constructs – do not use goto statements, setjmp or longjmp constructs, and direct or indirect recursion. Rationale: Simpler control flow translates into stronger capabilities for verification and often results in improved…

It's also important to note that these rules are made for critical control systems that tend to be low level. The cost benefit trade offs aren't going to be the same as in typical business software.

Re: How Is Critical Life or Death Software Tested?

#127

Post by mturmon »

acranox wrote: post #91
mlonkibjuyhv wrote: post #61

Earlier quoted context omitted.

Cars often disable any system capable of interfering with the wheels at the least sign of issue. Insufficient seal on the fuel-cap? Disable ABS, TCS, etc.

That's not correct. Cars will generally have something that's often called a 'limp home' mode. I saw a car that's PCM (powertrain control module) that failed some internal test, so it fell back to a basic mode where the engine wouldn't rev over about 2000RPM, and I'm sure all the emissions systems went into a basic fail-safe mode, where things like the fuel delivery goes into a hard-coded mode, instead of using feedb…

That's neat. The counterpart for spacecraft is "safe mode" (http://en.wikipedia.org/wiki/Safe_mode_(spacecraft) ). The priorities are maintaining attitude control, conserving power, and listening to the radio. The science instruments and other bells and whistles are turned off as much as possible.

Re: How Is Critical Life or Death Software Tested?

#128

Post by [deleted] »

acomjean wrote: post #8

I wrote software for Radars. Kind of important (not like plane software). We used Ada alot, which in my estimation helped. Software was reviewed. Tests were reviewed. Reliability was favored over other things (for example recursion was discouraged). We used Ada's constrained types (this value is between 1 and 99, if it goes out of range, throw an exception). For external hardware inputs, we had software simulating th…

[deleted]

Re: How Is Critical Life or Death Software Tested?

#130

Post by neurotech1 »

infinotize wrote: post #101
neurotech1 wrote: post #98

Earlier quoted context omitted.

An F-22 had a FCS failure and didn't realize takeoff. The pilot didn't do a IBIT on the FCS as required, therefore wasn't aware that all 3 rate sensors latched up. The jet was uncontrollable once it left the ground, and the pilot ejected safely. [0] As I recall, they modified the sensors to avoid latch up (extra pullup resister) and updated the FCS software to provide a warning if all 3 sensors return zero output. Ev…

To be fair this seems to be more of a classic example of a documentation or training problem. Right from your link: > During the mishap sequence, the MP started engines, perfomled an IBIT, and had a fully functioning Flight Control System. Subsequently, the MP shut down engines to allow maintenance personnel to service the Stored Energy System. During engine shut down, the MA's Auxiliary Power System (APU) was runnin…

True. The pilot wasn't the only one who misunderstood the details.
Post reply on HN