How Is Critical Life or Death Software Tested?
motherboard.vice.com
How Is Critical Life or Death Software Tested?
1–10 of 161 posts
Re: How Is Critical Life or Death Software Tested?
#2[deleted]
Re: How Is Critical Life or Death Software Tested?
#3It's essentially an analog of the concept of tolerance in the physical world of manufacturing and assembly. The less your tolerance for error the more formal and carefully controlled the process, and the more money spent in testing, verification, feedback, and improvement.
And yet you can still measure one value in the metric system and another in English units and drill a smoking hole in Mars. It was sort of striking to read Charles Fishman's statement about the software being bug free, followed immediately by the supporting fact that the last three versions had one bug each. If they had one bug, how are you 100% sure they didn't have two?
Re: How Is Critical Life or Death Software Tested?
#4It's essentially an analog of the concept of tolerance in the physical world of manufacturing and assembly. The less your tolerance for error the more formal and carefully controlled the process, and the more money spent in testing, verification, feedback, and improvement. And yet you can still measure one value in the metric system and another in English units and drill a smoking hole in Mars. It was sort of strikin…
> It was sort of striking to read Charles Fishman's statement about the software being bug free, followed immediately by the supporting fact that the last three versions had one bug each. If they had one bug, how are you 100% sure they didn't have two?
I think you're being overcritical here. It's really really really difficult to reach 100% in any real form of measurement. So I think when they mean "bug free", they probably mean the chance of a bug is below some threshold of probability. The famous six sigma rule comes in mind.
I do grant you that this is not specified explicitly in the article, but they do say: "each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors."
Re: How Is Critical Life or Death Software Tested?
#5It's essentially an analog of the concept of tolerance in the physical world of manufacturing and assembly. The less your tolerance for error the more formal and carefully controlled the process, and the more money spent in testing, verification, feedback, and improvement. And yet you can still measure one value in the metric system and another in English units and drill a smoking hole in Mars. It was sort of strikin…
Bugs? You mean features!
Re: How Is Critical Life or Death Software Tested?
#6It's essentially an analog of the concept of tolerance in the physical world of manufacturing and assembly. The less your tolerance for error the more formal and carefully controlled the process, and the more money spent in testing, verification, feedback, and improvement. And yet you can still measure one value in the metric system and another in English units and drill a smoking hole in Mars. It was sort of strikin…
> It was sort of striking to read Charles Fishman's statement about the software being bug free, followed immediately by the supporting fact that the last three versions had one bug each. If they had one bug, how are you 100% sure they didn't have two? I think you're being overcritical here. It's really really really difficult to reach 100% in any real form of measurement. So I think when they mean "bug free", they p…
Because it is so difficult you should not make statements that contain fragments such as 'bug free' unless they are preceded by a negation.
Re: How Is Critical Life or Death Software Tested?
#7Could have at least mentioned DO-178C
Re: How Is Critical Life or Death Software Tested?
#8I wrote software for Radars. Kind of important (not like plane software).
We used Ada alot, which in my estimation helped. Software was reviewed. Tests were reviewed. Reliability was favored over other things (for example recursion was discouraged). We used Ada's constrained types (this value is between 1 and 99, if it goes out of range, throw an exception).
For external hardware inputs, we had software simulating the messages when we couldn't get the real equipment in the lab. We were writing software for a radar that didn't yet exist.
And it was tested and tested and tested and tested.....
The when the pieces came together in integration. They tested and tested. An when they built the physical radar, the software worked. A few configuration things here and there needed adjusting.
Actually testing was built into the software. When it came up it would talk to the physical parts to make sure everything was communicating ok before it could start running.
Was it bug free? Probably not. You can't test every possible scenario. But we took the work seriously. It took longer to get code done than any other environment I've been in, but the code ended up very solid quality.
Re: How Is Critical Life or Death Software Tested?
#9I worked on an emr.
Each developer got between 1/2 to 3/2 qa people, in addition to dedicated qa engineers. You had to submit detailed test plans for features. Surprisingly, the company didn't expend much effort on unit tests -- they were there, but not heavily emphasized.
Re: How Is Critical Life or Death Software Tested?
#10It's all just a question of cost.
We know how to write software that comes arbitrary close to perfection. But as defects asymptotically approach zero, cost skyrockets.
The interesting question is what technologies can bend that cost/quality curve.