How Is Critical Life or Death Software Tested? • Hacker News Board

Re: How Is Critical Life or Death Software Tested?

#111

Post by eropple » Tue, Jun 02, 2015, 2:22 AM UTC

robotresearcher wrote: post #93

fredmorcos wrote: post #62

Earlier quoted context omitted.

> Reliability was favored over other things (for example recursion was discouraged). This sounds really strange to me. So may I ask why? I find that recursion - most of the time - helps shorten and clarify the code. Also, doesn't recursion make induction proofs trivial?

tldr version: Because you want (i) an acyclic function call tree of (ii) predictable depth.

That reminds me--a friend was telling me earlier today about a piece of software he was working on that only allowed forward jumps. (It ensures that the program halts.)

Re: How Is Critical Life or Death Software Tested?

#112

Post by HeyLaughingBoy » Tue, Jun 02, 2015, 2:36 AM UTC

Kluny wrote: post #83

jerf wrote: post #23

Earlier quoted context omitted.

This is why this discussion sometimes frustrates me. A lot of the defects we have are because you aren't willing to pay for the sort of software that wouldn't have defects. It's natural to read that as a sort of cynical accusation, but instead, I mean it straight... you really aren't willing to pay what it would take, and you shouldn't be. A $1000 Facebook-access app for your phone (that still somehow has some sort o…

Keep in mind that this level of testing sometimes isn't available at any cost - my company, for example, even if we were awarded a million-dollar-plus contract for the product we already build, would not be able to come up with the stringent testing that those NASA engineers use.

You learn. Or farm it out to the people who know.

Re: How Is Critical Life or Death Software Tested?

#113

Post by jacobsenscott » Tue, Jun 02, 2015, 2:41 AM UTC

aswanson wrote: post #103

noir_lord wrote: post #69

Earlier quoted context omitted.

> The parts I do have experience with it is utterly a miracle that anything works. Is my feeling about everything industry I've worked in. The stuff that runs telecoms (mostly billing side) particularly is the stuff of nightmares.

Seriously? Billing code? I would have imagined that code would be so subject to customer complaint that it would be forced into quality.

I would imagine this is a part of the code everyone is terrified to touch. When bugs do turn up they would be fixed by tactical 'if' statements. It would be a shit pile of hacks built up over many years.

Re: How Is Critical Life or Death Software Tested?

#114

Post by andrewchambers » Tue, Jun 02, 2015, 2:46 AM UTC

aswanson wrote: post #103

noir_lord wrote: post #69

Earlier quoted context omitted.

> The parts I do have experience with it is utterly a miracle that anything works. Is my feeling about everything industry I've worked in. The stuff that runs telecoms (mostly billing side) particularly is the stuff of nightmares.

Seriously? Billing code? I would have imagined that code would be so subject to customer complaint that it would be forced into quality.

support staff are probably manually correcting problems when someone calls up.

Re: How Is Critical Life or Death Software Tested?

#115

Post by TheLoneWolfling » Tue, Jun 02, 2015, 2:47 AM UTC

derekp7 wrote: post #100

dugmartin wrote: post #64

Earlier quoted context omitted.

Early in my career I worked on military flight data recorders, including the development of the software for the F-22's "black box". Those systems have SBIT, IBIT, PBIT and MBIT sub-systems were BIT is "built in test" and S = startup, I = initiated, P = periodic and M = maintenance. I remember making the Star Trek diagnostic joke myself when I was assigned the SBIT work. Each BIT does varying level of testing based o…

What types of errors would cause one of the tests to fail? Is it mostly testing for hardware errors, or are there any software logic errors that could make it to production, but be caught by one of the tests several months down the road? The only software related items I can think of are edge cases where a built in test is based on real time input. Kind of like running the calculations through multiple independent im…

Another thing that's software-related is if you've got a (rare) race condition. For example a data structure that gets corrupted if you have a particular series of nested interrupts.

Now, hopefully your system is set up such that race conditions cannot happen, but good luck with that.

Re: How Is Critical Life or Death Software Tested?

#116

Post by zobzu » Tue, Jun 02, 2015, 2:50 AM UTC

"There's a serious move towards formal verification rather than random functional testing," he writes. "Government agencies like NASA and some defense organizations are spending more and more on these technologies. They're still a PITA [pain in the ass] for the average programmer, but they're often more effective at testing critical systems."

this part is kinda cool because there's also always been a movement for toward this for opensource/commercial security software, at least for the kernel. Of course, we have no such thing today (as in i doubt anyone here runs such a system in production), but the interest is there.

Re: How Is Critical Life or Death Software Tested?

#117

Post by TheLoneWolfling » Tue, Jun 02, 2015, 2:51 AM UTC

eropple wrote: post #111

robotresearcher wrote: post #93

Earlier quoted context omitted.

tldr version: Because you want (i) an acyclic function call tree of (ii) predictable depth.

That reminds me--a friend was telling me earlier today about a piece of software he was working on that only allowed forward jumps. (It ensures that the program halts.)

It does slightly more than that - it provides a cheap computation of the upper bound of how long it will take.

Re: How Is Critical Life or Death Software Tested?

#118

Post by TheLoneWolfling » Tue, Jun 02, 2015, 2:55 AM UTC

manarth wrote: post #43

Aleman360 wrote: post #37

Worked on anesthesia machines for a few years. Since both hardware and software is involved, the testing was quite extensive. * Lots of manual testing. While we did unit testing and some automated integration testing, most defects were found using exhaustive manual testing by trained engineers. * Randomized UI testing. Used UI automation to exercise the UI with various physical configurations of the system. Would oft…

You just have to look at Therac 25 for the risks of relying on software interlocks alone. One of the prevailing pieces of feedback was the lack of hardware interlocks - whether that's possible on anesthesia machines I don't know…but the prevailing wisdom is to use/include hardware interlocks wherever that's feasible, for any critical life-supporting equipment.

The other thing is to not ditch the software interlocks just because there are hardware interlocks. And never ignore interlock triggers.

Re: How Is Critical Life or Death Software Tested?

#119

Post by eropple » Tue, Jun 02, 2015, 2:56 AM UTC

TheLoneWolfling wrote: post #117

eropple wrote: post #111

Earlier quoted context omitted.

That reminds me--a friend was telling me earlier today about a piece of software he was working on that only allowed forward jumps. (It ensures that the program halts.)

It does slightly more than that - it provides a cheap computation of the upper bound of how long it will take.

Yeah, that too. Come to think of it, that was probably more his concern than just halting.

Re: How Is Critical Life or Death Software Tested?

#120

Post by aaronharnly » Tue, Jun 02, 2015, 2:59 AM UTC

cesarbs wrote: post #17

> As for the code itself, its perfection came as the result of basically the opposite of every trope normally assigned to "coder." Creativity in the shuttle group was discouraged; shifts were nine-to-five; code hotshots and superstars were not tolerated; over half of the team consisted of women; debugging barely existed because mistakes were the rarest of occurrences. Programming was the product not of coders and eng…

This type of coding is I think best understood by the following tradeoff: they know exactly what the code must do; and the loss if it fails to do that thing is enormous; so it worth expending enormous effort to ensure it does that thing exactly.

In most user-facing software in the Internet age, the reverse is true: we do not really know what the software should do; but the penalty for a bug is not great; so, it is not worth expending enormous effort to be big free; instead it is better to expend great effort to be nimble and find out just what it is the software should be doing to begin with.

Very different environments yielding very different methodologies. I can't say one is better, just de gustibus non est disputandum.