How Is Critical Life or Death Software Tested? • Hacker News Board

Re: How Is Critical Life or Death Software Tested?

#151

Post by airza » Tue, Jun 02, 2015, 2:14 PM UTC

mbrameld wrote: post #143

nchelluri wrote: post #102

Earlier quoted context omitted.

Care to elaborate on the potato chip bag story? Sounds intriguing.

My wild guess loosely based on my time spent writing software to analyze observed radar signals: I believe the parent probably opened the bag along the seams to get one flat piece of reflective (on the inside of the bag) material. I know weather balloons are often reflective so they can be tracked by radar. I'm guessing a full-size potato chip bag opened flat would be big enough for the radar to see, couldn't tell yo…

This is pretty much it. My boss was definitely a level 99 duct tape programmer, and this was one of the many examples of his ingenuity.

Re: How Is Critical Life or Death Software Tested?

#152

Post by sacado2 » Tue, Jun 02, 2015, 2:40 PM UTC

peteretep wrote: post #142

z3t4 wrote: post #137

Earlier quoted context omitted.

One mistake that ppl do is they wrap their code around a try ... catch, where it's better to throw an error and exit. If there's an error in one place, chances are there are also errors elsewhere, so it's better to restart the program instead of continue with a bad state. When the error gets thrown in your face, there's a higher chance that it gets fixed. But this also have its setbacks. Loosing the whole state can b…

I'm really note sure why you think catching an error in a separate process is somehow superior to catching it in a higher scope

I think it depends on the kind of error. If it is a "bug-detected" error (null-pointer dereference, out-of-bounds, divide-by-zero, out-of-memory, etc.), you better restart the program since you're in an unstable state. If it is a "domain-specific" error (connection lost, robot could not reach its destination, battery low, etc.), you better deal with it as soon as possible.

Re: How Is Critical Life or Death Software Tested?

#153

Post by annnnd » Tue, Jun 02, 2015, 2:48 PM UTC

gtirloni wrote: post #146

aswanson wrote: post #103

Earlier quoted context omitted.

Seriously? Billing code? I would have imagined that code would be so subject to customer complaint that it would be forced into quality.

I've worked in a support team for a telecom billing system and I was tasked with interacting with our development team to investigate bugs in production (and eventually moved to the development team). These systems are created just like any other commercial system, without any formal proofs and minimum requirement docs. To make things worse, they have to be flexible enough to support all and any billing plans that th…

It is a nice challenge how to rewrite such a system. One way would be to build a parallel system, identify all inputs and duplicate them to the parallel solution and then compare the outputs; in case of discrepancies fix the erroneous system. Once the systems produce same results (or once the new system produces better results than the old one) you just switch the systems.

The rewrite doesn't have to be complete; it can (should) be done in pieces of course.

Re: How Is Critical Life or Death Software Tested?

#154

Post by digikata » Tue, Jun 02, 2015, 3:48 PM UTC

mitchty wrote: post #97

manarth wrote: post #86

Earlier quoted context omitted.

"ship both a and b, whose output always has to match exactly or it reports a fault and the component requires replacement." And when that software disagrees…but the plane happens to be at 40,000 feet? The plane just stops running until the component is replaced? I don't know too much about plane hardware, but I scuba-dive a rebreather which has critical life-support electronics. It has two independent computers, and…

I have a friend that used to do embedded programming for nuclear power plants. From what he said they have 5 systems, two running software from vendor a, two from vendor b, and another running software from vendor c for failsafe. A quorum have to agree, if any don't it gets pulled out of service and analyzed why it wasn't the same output. If you get down to one there is still a mechanical failover but at that point y…

Who writes the software for the quorum? Or is it hardware?

Re: How Is Critical Life or Death Software Tested?

#155

Post by dugmartin » Tue, Jun 02, 2015, 5:08 PM UTC

michaelt wrote: post #145

infinotize wrote: post #101

Earlier quoted context omitted.

To be fair this seems to be more of a classic example of a documentation or training problem. Right from your link: > During the mishap sequence, the MP started engines, perfomled an IBIT, and had a fully functioning Flight Control System. Subsequently, the MP shut down engines to allow maintenance personnel to service the Stored Energy System. During engine shut down, the MA's Auxiliary Power System (APU) was runnin…

To be fair this seems to be more of a classic example of a documentation or training problem. Maybe - but if you were designing a consumer product, you wouldn't rely on the user following a checklist; the IBIT would run automatically when they turned on the ignition and sound an alarm (or even prevent takeoff) if the vehicle would be uncontrollable. So you could also classify this as a user interface / design problem…

SBIT & PBITs are the ones that run automatically, with SBIT running automatically at startup and PBIT running on a watchdog. The SBIT time budget and scope is usually much smaller than IBIT so time intensive tests like ones that talk to sensors on the bus aren't present in SBIT. You can think of the stages as SBIT: can I run? IBIT: should I run? PBIT: am I running right?

Re: How Is Critical Life or Death Software Tested?

#156

Post by na85 » Tue, Jun 02, 2015, 6:47 PM UTC

ufmace wrote: post #122

superuser2 wrote: post #33

Earlier quoted context omitted.

No. That it contained under n ppm of impurities, that it did not leak more than 0.001%/year, maybe. Only a Sith (and consumer product marketing that is lying to you) deals in absolutes. In the durable goods world, you don't pretend things are perfect. Failure modes are designed and disclosed, replacement of parts is expected and made reasonable, tolerances are marked, failure rate metrics like MTBF are known, and as…

That reminded me of phone plans, oddly enough. For a while, everybody was advertising their data plans as unlimited, and then the tech press would get all upset when they found the limit. I always thought the whole thing was dumb - of course there's a limit. Or are we supposed to believe we can push megabytes/s nonstop all month? I'd rather have them just tell me what the limit is and what happens when you go over it…

You're okay with scummy advertising that probably breaks false advertising laws?

Re: How Is Critical Life or Death Software Tested?

#157

Post by noir_lord » Tue, Jun 02, 2015, 11:42 PM UTC

annnnd wrote: post #153

gtirloni wrote: post #146

Earlier quoted context omitted.

I've worked in a support team for a telecom billing system and I was tasked with interacting with our development team to investigate bugs in production (and eventually moved to the development team). These systems are created just like any other commercial system, without any formal proofs and minimum requirement docs. To make things worse, they have to be flexible enough to support all and any billing plans that th…

It is a nice challenge how to rewrite such a system. One way would be to build a parallel system, identify all inputs and duplicate them to the parallel solution and then compare the outputs; in case of discrepancies fix the erroneous system. Once the systems produce same results (or once the new system produces better results than the old one) you just switch the systems. The rewrite doesn't have to be complete; it…

"Replace in place" is fairly common.

The big issues is duplicating the system while it is still morphing in production for all the edge cases, it often feels like trying to paint a moving bus.

Re: How Is Critical Life or Death Software Tested?

#158

Post by takeda » Wed, Jun 03, 2015, 5:55 AM UTC

_nedR wrote: post #133

mlonkibjuyhv wrote: post #61

Earlier quoted context omitted.

Cars often disable any system capable of interfering with the wheels at the least sign of issue. Insufficient seal on the fuel-cap? Disable ABS, TCS, etc.

What really scares me are the ABS and traction control systems that are now becoming mainstream on motorcycles. These systems have accelerometers and gyros, which take in to account, information like lean angle of the bike when calculating the braking force. It certainly is a feat of engineering. Little about being a biker, programmer, and an aspiring tree-shade mechanic, reassures me about the safety of these system…

> although stuff like Toyota's unintended acceleration does not inspire confidence

Another thing that worries me are that these same companies are also working on self driving cars.

Worst of it is that everyone pretty much jumped into the race after Google. It also doesn't look like existing solutions work on a real time system. I'm a bit worried about being hit by a car because it was running garbage collection process and did not react quickly enough.

Re: How Is Critical Life or Death Software Tested?

#159

Post by dbdr » Wed, Jun 03, 2015, 5:57 PM UTC

henrik_w wrote: post #141

The story of the Boeing engineers flying on the test flights is a perfect example of "skin in the game" (from Antifragile by Nassim Nicholas Taleb). Here's what I wrote about that in a blog post on Antifragility and SW development: At the end of the book, there is a chapter on ethics that Taleb calls “skin in the game”. To have skin in the game, you should share both in the upside and downside. Taleb quotes the 3,800…

The examples you are giving are all about sharing the downside. What are the options for the upside?

Re: How Is Critical Life or Death Software Tested?

#160

Post by crucini » Thu, Jun 04, 2015, 4:47 PM UTC

I think the right answer here is to develop languages or libraries that enforce the constraints you want.

I think we should beware of adding layers of "silver bullet" technologies that promise to fix the last crisis, but increase the friction of development.

To illustrate what I mean, PLCs are normally programmed in ladder logic. From that level, I don't think you can crash the machine or corrupt memory. So the risks are limited to the "operating system" if you will, which can be more mature and tested than the "application".