Live data from Hacker News

Failed intercept at Dhahran caused by a software error in handling of timestamps

en.wikipedia.org

61–70 of 91 posts

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#61
post #46

This was a tragic and preventable loss. It's incredible that a software bug might have been the root cause. At the time, this incident really stuck out because it broke the illusion of our fabled Patriot missile shield protecting us. Civilian expats really believed the inflated Patriot interception rates parroted to us by mainstream media and our American military expat buddies. A large number of remaining expats who…

I mean I kind of understand the attitude of the British Embassy, it wasnt like trouble flared up overnight, the option to leave was there for a long time prior to the war beginning. Obviously it isnt the fault of the children who were kept there by their parents, but some responsibility needs to be borne by the expats that decided they were getting paid well enough to stay.

We all understood their stance. The notable point is the stark contrast between the Americans (embassy and expats alike) and everybody else.

While most of us were cowering under our desks and tables during SCUD attacks, some of our American civilian friends were out with their families in the desert trying to film the Patriots "intercepting" the SCUDs and driving out to try and pick up pieces of debris.

I look back upon those days with fondness and gratitude, especially for the American forces that served.

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#62

I could be reading this wrong, but 1/3 of a second within 100 hours seems really good, like something you'd get from a temperature-controlled crystal oven. I don't mean to second-guess them in an area I know so little about, but if that was enough to cause a serious issue in the span of only a few days, shouldn't the devices be designed with a separate synchronization system, at least as a backup? Maybe GPS? Which br…

It wasn’t clock drift, it was an error in calculation leading to separate parts of the system, that were calibrated to the same common clock, to drift out of synchronization. Using a different clock, like GPS wouldn’t help with this.

But the rest of your point boils down to ’if you know your system has a flaw why not mitigate it’? But of course at design time they didn’t know it had this flaw.

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#63
post #31

The title of this post is misleading, they eventually supplied a software patch that fixed the clock drift. The Israelis proposed rebooting as a stopgap until the bug could be fixed.

We've updated the submitted title from “Clock error lead to death of 28 Soldiers. Software fix: Reboot system regularly” to a representative phrase (edited for length) from the article. Submitters: please follow the guidelines by not editorializing titles.

https://news.ycombinator.com/newsguidelines.html

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#64
post #56

This is a completely misleading headline. The Patriot missile was not effective at destroying the Scud [0]. The DoD initially claimed successful intercepts when the missile detonated near the Scud, but it rarely, if ever, actually destroyed the warhead. The only reason there was an illusion of success was that the Scud was also spectacularly unreliable and often broke up on re-entry or failed to detonate. It is a com…

[deleted]

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#65
This particular bug is often taught in university compsci classes as "bug that killed people" is a good attention grabber -- the CS/EE analysis is sound; its truthfulness is only suspect because of the DoD's claimed successes.

A more truthful "computer bugs that killed people" example would be the Therac-25 - a machine intended to treat cancer with tightly-focused radiation therapy. Six patients died as a result of massive overdoses of radiation, on the order of 20,000 rads. It was possible for the machine to end up in a state where it delivered full-power radiation without a hardware shield in place to protect the rest of the patient's body. No hardware interlocks were used to ensure that the full power mode was only usable with the shield in place - all safety features relied on software. In addition, the bug was only possible when an operator made a mistake in mode selection and then rapidly (proficiently) corrected it - the rapidity required prevented the bug from being discovered during slow, methodic, careful testing.

See Hackaday's article Killed by a Machine (and associated HN discussion) or for the especially curious, a 49-page post-mortem for more detail:

https://hackaday.com/2015/10/26/killed-by-a-machine-the-ther...

https://news.ycombinator.com/item?id=12201147

http://sunnyday.mit.edu/papers/therac.pdf

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#66
post #8

Earlier quoted context omitted.

Oh, gosh. Really? It makes me kind of sick imagining that call.

You're not even curious to see how it was dealt with and how the issue was expressed to the vendor? I'd never be in a meeting regarding deaths of users of my software, because I just make internal webapps, so I just cannot help but be curious as to how one of those meetings would go.

> I'd never be in a meeting regarding deaths of users of my software

I know what that's like.

About 20 years ago I was at a consulting firm supporting an electric and gas utility company. Among other things they had to do something called "markouts" which means they paint the ground at a location in a way that indicates exactly what infrastructure they have in the ground and precisely where it is. Markouts are a government organized thing. Before digging somewhere you can call a number and anybody that might possibly have infrastructure in the ground anywhere near your dig site is required to paint their markouts within a short time period. There are stiff fines if you "miss a markout."

Anyhow there was a data problem with a markout. The field worker was sent to paint a markout at the corner of two streets that actually ran parallel to each other and didn't meet. Instead of calling it in and questioning the task he did nothing. Shortly after a construction worker put a backhoe through an electrical conduit with 15K volts. There was an explosion that was heard for many miles. The worker died the next day. He died painfully.

> so I just cannot help but be curious as to how one of those meetings would go.

Finger pointing, of course. Data was being fed back and forth between systems and eventually somebody else took the blame. The field worker who ignored the markout also was blamed. We did add something to our system so that that kind of data error would raise an exception.

I learned a lot about care and diligence about data from this experience. Data errors are no joke.

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#67
post #8

Earlier quoted context omitted.

You're not even curious to see how it was dealt with and how the issue was expressed to the vendor? I'd never be in a meeting regarding deaths of users of my software, because I just make internal webapps, so I just cannot help but be curious as to how one of those meetings would go.

Sure but > I'd have loved to be a fly ... loving anything about that sad scenario seems impossible.

Even loving learning enough to avoid the next one? 'Cause that's what responder wants, to learn. "What the hell were they thinking?" is often the most pertinent knowledge of all.

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#68
post #13

Earlier quoted context omitted.

I learned about it in a Decision Analysis course and had a completely different point driven home. This wasn't bad code. It was code that was correctly written to a very well defined requirement ("System shall be operational for at most X hours before a reboot"). The code was written to a spec that was approved by the customer (the military). Unfortunately though, that requirement wasn't communicated to the end users…

I'm failing to find anything that says the requirement was "System shall be operational for at most X hours before a reboot". It's more likely that there was a key performance paramater (KPP) saying that it should be functional for at least some period of time. And that was what was tested. Generally KPPs (which aren't requirements themselves, but influence the requirements for systems) are set at lower bounds, not u…

Its interesting that FM 44-85 "Patriot Battalion and Battery Operations" is publicly available and pretty easy to find. We discussed this in a systems analysis class back in '04 using a copy of FM 44-85 released in '97. In summary the class blamed TRADOC and the tech writers for publishing a manual that did not accurately reflect real world use cases, with the software bug being a secondary concern.

I googled up a copy of FM 44-85 to refresh my memory and write this post, its pretty much as I remember it.

The doctrine in chapter 3, planning, is extreme mobility and rapid hour to hour activation and deactivation of individual missile batteries, kinda like infantry bounding overwatch but glacially slower on an hourly basis, for example see Table 3-2 where the four batteries are rotating on and off and moving/maintaining on detailed hour by hour basis, so the doctrine seems to be uptimes should typically be on the order of 3 or 4 hours maybe. Not a zillion days in a row as actually deployed when the software bug hit.

The doctrine in chapter 5, operations, goes into a big discussion of defense design strategies. The weapon system is inherently sectorized this naturally leads to overlapping areas of fire being very important. You have to ask why the unit that had a ridiculous uptime never shut down to perform daily maintenance which would inherently involve rebooting stuff, its no big deal to down a system because sectorization and overlap is inherently built into the technology. Its reasonably well understood that technically you can tell an individual infantry soldier to guard a post for 100 hours or 1000 hours continuously, but someone screwed up if they issued an order like that because its simply impractical. That leadership failure will be discussed later. So... aside from the question of why the software failed under ridiculous conditions, you have to ask WHO more or less knowingly misapplied the resource without backup or planned maintenance intervals? Possibly this section of the FM was rewritten between the tragedy and the the release of the copy I have access to, but its still poorly written. Or what section of the FM would have ever given the officers the idea that the weapons system can be deployed the way they did it? The idea that the weapon system could do what they told it to do came from somewhere and it apparently was not the documentation?

The doctrine in chapter 6, support, has a little blurb about battalion level staff officers. What did the EMMO think about keeping a patriot booted up and running for 100 hours without a maintenance interval? Missile maintenance is literally his only job. And if that slot was unfilled, its the job of S4 and the XO to cover or reassign someone or otherwise work around. Around page 6-16 there's a discussion about operators being responsible for maintenance... I had a humvee assigned to me, I hated it, it leaked oil all the time, but the point is even my junky humvee had daily maintenance tasks for PMCS. The patriot missile PMCS checklist is probably classified, but if a lowly humvee has daily maint, how can a missile not have a much longer and more complicated daily maint? And this implies someone is pencil whipping maint (I mean, everyone kinda does that, but..)

Its hard to summarize a class discussion but from the point of view of a systems analysis class, mostly non-military other than myself, the end users were being innovative and adapting and overcoming which unfortunately means the doctrine and specifications of the weapon system have little to do with how its being used. The class considered this the biggest systems analysis mistake of the tragedy. Why even write docs and specs if the users won't read them and they have no relation to what the users want to do? I guess a good HN analogy would be you could creatively deploy binary executables using the "cat" command and hand typing unicode and that would be a nifty hack to work around a problem but would be a pretty stupid way to operate normally. Specifically the Army's own docs used to train and plan operations imply shorter operations terms interspersed with maintenance intervals and deep redundancy, none of which seems to have anything to do with the failed deployment.

There was a big argument in class that it had nothing to do with systems analysis and was merely a leadership failure, using the example above of technically you can order a soldier to stand guard for a hundred hours, when guard shifts are normally a couple hours, and when he passes out asleep around 48 hours into his shift, you can try to blame the soldier or declare there's a bug in our brain preventing 100 hour deployments, or you can even blame the manual and the technical writers for not putting a warning in the manual not to do dumb things, but fundamentally thats just passing the buck that it was a failure of leadership to assign a unit to a task its not designed to handle, then cover it up by pretending its merely a software bug or something. I don't know enough history of the tragedy; its possible the Army correctly relieved some officers of command and its only the media and press who blame the software bug.

You can imagine the look on the face of the software developers when they got the bug report; like dude, did you ever read FM 44-85, or if you aren't reading it, what are you reading, so we can read it?

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#69
post #62

I could be reading this wrong, but 1/3 of a second within 100 hours seems really good, like something you'd get from a temperature-controlled crystal oven. I don't mean to second-guess them in an area I know so little about, but if that was enough to cause a serious issue in the span of only a few days, shouldn't the devices be designed with a separate synchronization system, at least as a backup? Maybe GPS? Which br…

It wasn’t clock drift, it was an error in calculation leading to separate parts of the system, that were calibrated to the same common clock, to drift out of synchronization. Using a different clock, like GPS wouldn’t help with this. But the rest of your point boils down to ’if you know your system has a flaw why not mitigate it’? But of course at design time they didn’t know it had this flaw.

Good point - hindsight is 20/20...

Re: Failed intercept at Dhahran caused by a software error in handling of timestamps

#70

Earlier quoted context omitted.

I'm failing to find anything that says the requirement was "System shall be operational for at most X hours before a reboot". It's more likely that there was a key performance paramater (KPP) saying that it should be functional for at least some period of time. And that was what was tested. Generally KPPs (which aren't requirements themselves, but influence the requirements for systems) are set at lower bounds, not u…

As I gather a the Patriot was a mobile anti-aircraft / anti-cruise missile platform that was meant to move, be activated when needed, and then be turned off and move again because the original location was expected to become a target. It was pressed, on short notice (with some software upgrades, but not the normal cycle of specs, development, and validation that would go into that kind of repurposing) into stationary…

There's a distinction to be made, though. There was no requirement that it be rebooted after some period of time, though there was an expectation that this would happen by the original developers. Consequently it was not evaluated for 20 hour or 100 hour performance. That's a critical distinction in developing, testing, and fielding systems. And the way we term it in our requirements documents reflects this. We rarely say: System SHALL fail after some period. Rather we say: System SHALL perform for some period. We leave the result of longer durations undefined. The system may work, or it may not, we aren't required to test it and so we don't. If the customer wants it to run longer, we can evaluate it but they have to communicate that back to us (or to the testing facilities, which may not be the developers).

Similarly, with regards to the speed of the missiles, the requirement would not be: System SHALL fail to detect missiles above some threshold speed. But rather: System SHALL detect missiles below some threshold speed. This leaves open the possibility that it may be more or less accurate outside that range. It should be documented for the operators as a potential for failure: System may be ineffective against missiles operating above X m/s. But the requirements wouldn't include that detail.

This pushes the problem into the documentation and training. Since it was originally designed as a mobile platform with short run-times, there was no explicit operating procedure requiring reboots. It was just assumed. At the same time, the failure itself (after 20 hours) was unknown because testing hadn't been done to see what would happen.

Post reply on HN