Greybeard Stories: The Jamming Gyro

Sure, writing tests can find design bugs. But that's not really the question at hand here. The specifics are that we have a clear and obvious requirement ("torpedo should self-destruct if it turns a full 360 degrees") that turns about to be missing an important point ("EXCEPT IF IT IS ON THE BOAT").

No amount of testing the former will lead you to realize the latter. Sure, you might happen to come to the realization while writing the test, but you might do so over breakfast too.

I'm not saying "don't do testing". I'm trying to point out that it has limits. The fact that you've written tests and they pass doesn't get you off the hook for design bugs.

Interesting (and depressing) story. On one of your posts you say:

"Sometimes because of the nature of my work I get to hear stories from a greybeard with a past that is, well, "interesting." And again, because of the nature of my work, or more accurately, because of the nature of their work, the stories can't be verified."

What's the nature of your work?

It sounds like one of these stories with educational/entertainment purposes rather than a description of an actual problem or event. I did find one reference -

"D. E. Stevenson attributes this story to Nancy Leveson, Software System Safety, STAR '93, Ontario at Darlington, Ontario. 1993.

A torpedo was designed to self-destruct if it turned 180 degrees. Unfortunately for the test ship the torpedo stuck in the tube and the captain turned the ship around for port "

It sounds mildly more plausible since it doesn't begin with the rather improbable torpedo design problem of 'submarines shooting themselves with their own torpedo'.

Still, if tasked with designing a reasonably safe torpedo, one of the very first things you're likely to come up with are 'armed' and 'safe' modes with the torpedo staying in 'safe' mode until it is about to be launched. The next obvious safety feature would be to make the torpedo return to 'safe' mode the moment significantly abnormal conditions are encountered - say, stuck in tube, wildly off-course, etc.

Adding a self destruct mechanism seems highly unsafe - there's the problem of the self-destruct mechanism malfunctioning and activating at an inopportune time. Additionally, if the torpedo has no idea where it is, the last thing you probably want is having it blow up - possibly near you or a friendly.

The actual stories of the difficulties developing WWII torpedoes are quite interesting and offer plenty of lessons in complex systems design, testing and deployment - see:

http://www.ww2pacific.com/torpedo.html

A design flaw that must have been particularly galling: "The conventional contact exploder was designed for the earlier, slower, 33 knot, Mk 13 torpedo. The newer, faster, 46 knot, Mk 14 torpedo had higher inertial impacts that would cause the firing pin to miss the exploder cap. " In other words, the more squarely you hit your target, the more likely the torpedo would fail.

Seems implausible to me. Rather than a self-destruct, it would be more plausible to just disable the explosive trigger altogether. If a torpedo fails to hit target for whatever reason (including, but not limited to the original problem) you don't necessarily want it to explode in some random place. Nor is it clear how, if this failure did occur with loss of all hands, we would ever have known about it.

Despite this, I take the point of the story to be that self-corrective failure detection mechanisms should not be capable of causing greater harm than the maximum plausible damage of the problem they were intended to correct.

I guess I can't argue with that, as it's essentially unfalsifiable. But it's worth pointing out that writing a unit test, which is the subject that started this discussion, is definitely not a time when you're "fully considering potential failure scenarios". Unit tests are narrow and focused, and aimed at verifying features.

What you're talking about is something I'd call white box QA. Which is valuable, though it's essentially just an extension of design, and has the same limits.

My broad point still stands: you can't "process" your way out of this with extra testing. Some bugs are just inherent, and stem from the fact that we're human.