Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Is Critical Life or Death Software Tested?

motherboard.vice.com

Post reply on HN
Print view View original

101–110 of 161 posts

Re: How Is Critical Life or Death Software Tested?

#101

Post by infinotize »

neurotech1 wrote: post #98
dugmartin wrote: post #64

Earlier quoted context omitted.

Early in my career I worked on military flight data recorders, including the development of the software for the F-22's "black box". Those systems have SBIT, IBIT, PBIT and MBIT sub-systems were BIT is "built in test" and S = startup, I = initiated, P = periodic and M = maintenance. I remember making the Star Trek diagnostic joke myself when I was assigned the SBIT work. Each BIT does varying level of testing based o…

An F-22 had a FCS failure and didn't realize takeoff. The pilot didn't do a IBIT on the FCS as required, therefore wasn't aware that all 3 rate sensors latched up. The jet was uncontrollable once it left the ground, and the pilot ejected safely. [0] As I recall, they modified the sensors to avoid latch up (extra pullup resister) and updated the FCS software to provide a warning if all 3 sensors return zero output. Ev…

To be fair this seems to be more of a classic example of a documentation or training problem. Right from your link:

> During the mishap sequence, the MP started engines, perfomled an IBIT, and had a fully functioning Flight Control System. Subsequently, the MP shut down engines to allow maintenance personnel to service the Stored Energy System. During engine shut down, the MA's Auxiliary Power System (APU) was running. The MP believed the APU provided continuous power to the Flight Control System, and therefore another IBIT after engine restart was unnecessary. This belief was based on academic training, technical data system description, and was shared by most F/A-22 personnel interviewed during the investigation.

Re: How Is Critical Life or Death Software Tested?

#102

Post by nchelluri »

airza wrote: post #32

My experience with testing avionics controllers was: Everyone seemed to have the correct idea (that bugs were basically Not Allowed.) The company set up enough testing so that those bugs were eventually eliminated. However, the difference between good projects and bad projects was mostly the amount of time and money that this took. My favorite was one where we had an entire test harness written in Python that could c…

Care to elaborate on the potato chip bag story? Sounds intriguing.

Re: How Is Critical Life or Death Software Tested?

#103

Post by aswanson »

noir_lord wrote: post #69
DannoHung wrote: post #48

Earlier quoted context omitted.

Financial institutions do not necessarily do things this way. Some parts might, but I don't have any experience with them. The parts I do have experience with it is utterly a miracle that anything works.

> The parts I do have experience with it is utterly a miracle that anything works. Is my feeling about everything industry I've worked in. The stuff that runs telecoms (mostly billing side) particularly is the stuff of nightmares.

Seriously? Billing code? I would have imagined that code would be so subject to customer complaint that it would be forced into quality.

Re: How Is Critical Life or Death Software Tested?

#104

Post by bargl »

snops wrote: post #92
david-given wrote: post #54

Earlier quoted context omitted.

C# is an odd choice for avionics software, surely? It's got a really heavyweight runtime and nondeterministic behaviour due to the garbage collector. I assume this is the satellite's application layer and isn't real time? ...and what are you running it on? I wasn't aware of any embedded operating systems which supported C#!

I'm guessing its actually running on a ground station, providing commands or analysing data from satellites, hence why its non-critical and the need has arisen for parallel computation, presumably to speed it up. Many ARM embedded systems can support C# with the open source .net micro framework ( http://www.netmf.com ), which doesnt require any OS, and was originally developed for Microsoft's SPOT watch. I haven't us…

Yeah it's a ground station application. So it isn't running on any embedded hardware. We'd have done something different for that.

Performance isn't the only thing we are considering. We want to get improved performance, but our old analysis code was extreemly hard to maintain, so that went into the decision as well. Honestly, I'd probably pick another language, but I wasn't on the project when it started.

I personally would have liked to do this with F# because of how functional it is at it's core, but that's cuz we have a lot of Microsoft expertise in house.

Also one thing with the engineering apps is that anything where engineers (not software engieners) don't have to learn a new language is going to be an easier sell.

Re: How Is Critical Life or Death Software Tested?

#105

Post by npatrick04 »

derekp7 wrote: post #100
dugmartin wrote: post #64

Earlier quoted context omitted.

Early in my career I worked on military flight data recorders, including the development of the software for the F-22's "black box". Those systems have SBIT, IBIT, PBIT and MBIT sub-systems were BIT is "built in test" and S = startup, I = initiated, P = periodic and M = maintenance. I remember making the Star Trek diagnostic joke myself when I was assigned the SBIT work. Each BIT does varying level of testing based o…

What types of errors would cause one of the tests to fail? Is it mostly testing for hardware errors, or are there any software logic errors that could make it to production, but be caught by one of the tests several months down the road? The only software related items I can think of are edge cases where a built in test is based on real time input. Kind of like running the calculations through multiple independent im…

There's actually a decent probability of memory corruption in space applications due to radiation. So in addition to checking communication across busses, application checksums are typically run continuously.

Re: How Is Critical Life or Death Software Tested?

#107

Post by clearf »

Tloewald wrote: post #30
outworlder wrote: post #15

Earlier quoted context omitted.

> Actually testing was built into the software. When it came up it would talk to the physical parts to make sure everything was communicating ok before it could start running. I've been wondering about this for a while. We tend to run unit tests, integration tests, whatever tests, while the software is in development. However, once it is in "production" (for whatever definition of production), usually no tests are pe…

The project I'm working on (for the Federal government no less, and very much not life and death) involves having test data in the production system and tests that run in the production system. (I was impressed by this when I found out.)

Make sure they're well separated! c.f. http://www.wsj.com/articles/SB100014240527487033765045754919...

Re: How Is Critical Life or Death Software Tested?

#108

Post by rodgerd »

manarth wrote: post #40
mariopt wrote: post #12

It depends on the software budget and managers. When I worked a company one the CEO's had an idea about adding a camera to road semaphores so that cars wouldn't stop is the road is empty. I asked: What happens if the sun light hits the camera too much? The guy laughed in my face and told I was being ridiculous. I left the company some months later for other reasons but It was pretty scary to me to hear such words at…

Genuinely, in road safety, the company typically considers and weighs the cost of fixing an issue, vs the cost of lawsuits in the event of death/injury. One paper on the topic talks about the Ford Pinto fuel system design: http://users.wfu.edu/palmitar/Law&Valuation/Papers/1999/Legg... The GM ignition-switch recall also sparked a similar debate: http://en.wikipedia.org/wiki/2014_General_Motors_recall So it's not unco…

Which is one reason people should be a lot more suspicious of "tort reform" than they are.

Re: How Is Critical Life or Death Software Tested?

#109

Post by jarrettch »

I've been wondering about this lately. My stepdad has heart failure and they put a heart pump in his chest. It regulates blood flow and settings can be changed, etc. Leave it to a dev to think "How much testing has gone into this thing?" Even one minor slip-up in his blood flow, either too high or too low, could mean a stroke and possibly death. Or, God forbid, the thing crashes somehow and stops working.

That machine is much easier to code than a space shuttle obviously, but I still wondered about it. The tech has to be rock solid. Even one malfunction could cause so much despair in a family, and could also cost your company millions.

Post reply on HN