How We Nearly Lost the Discovery Shuttle • Hacker News Board

Re: How We Nearly Lost the Discovery Shuttle

#11

Post by InclinedPlane » Fri, Aug 17, 2012, 8:04 PM UTC

aaronblohowiak wrote: post #7

InclinedPlane wrote: post #3

Every Shuttle in the fleet has had one or several extremely close scrapes with death. To look at the Shuttle record and see the history of calamity it's easy to think that we just had bad luck, but quite the opposite was the case. We were enormously lucky with the Shuttle, in a fairer world we would have lost more of them, and sooner. The Shuttle was plagued by many fundamental design flaws which combined to make it…

It was a 30-year design. Not too bad for our first reusable space plane.

At the risk of being a negative nelly, the Soyuz design is even older, and overall probably fundamentally safer. Also, the Shuttle is only ostensibly "reusable", in reality every single flight required months of refurbishment which included meticulous inspections of the TPS tiles, complete replacement of the payload bay liner, complete purging of all fuel in the system, and even replacement of the engines.

We certainly learned a lot from Shuttle operations but in terms of spacecraft design mostly we learned what not to do.

Re: How We Nearly Lost the Discovery Shuttle

#12

Post by masklinn » Fri, Aug 17, 2012, 8:05 PM UTC

geuis wrote: post #5

jevinskie wrote: post #2

"We informed the foam technicians at our plant in Michoud Louisiana that they were the cause of the loss of Columbia..." That to me is pretty disgusting. In an incident like the loss of Columbia, there is no one, true "root cause". To assign blame to those foam technicians was disingenuous and just another instance of "passing the buck" that seems to happen so often in the post-mortem of NASA failures. NASA knew of e…

Understand the type of organization NASA is. First, accountability is everything. It's not that there are big internal political struggles (there are), but more importantly accountability is required for high safety. We aren't talking about a 10 person startup where a bad commit to production takes down a site for a few hours. We're talking about people's lives, careers, and the safety of millions of people that coul…

> accountability is required for high safety

It's not. That's complete bullshit, and the On-Board Software Group demonstrated it by being as flawless as can be during the whole history of the Shuttle: as far as I know there was no personal accountability in the OSG, the only thing accountable was The Process supported by a strong culture of adversarial testing.

Personal accountability in such a system brings politics and career advancement in focus and leads to issues being shoved under the rug when inconvenient and energy being expended in finger-pointing and blame games rather than fixing problems.

> So it's not a personal, vindictive "your fault", its an impersonal "the problem is here, lets fix it".

No, it's not. Accountability is very precisely "your fault", that's all it is. That's pretty much the definition of it.

Re: How We Nearly Lost the Discovery Shuttle

#13

Post by vladd » Fri, Aug 17, 2012, 8:07 PM UTC

masklinn wrote: post #10

InclinedPlane wrote: post #3

Every Shuttle in the fleet has had one or several extremely close scrapes with death. To look at the Shuttle record and see the history of calamity it's easy to think that we just had bad luck, but quite the opposite was the case. We were enormously lucky with the Shuttle, in a fairer world we would have lost more of them, and sooner. The Shuttle was plagued by many fundamental design flaws which combined to make it…

In that, the Shuttle was very much like the Concorde: a unique and complex system beyond the edge of knowledge (at its creation), full of flaws and working through a combination of sheer luck and heroic efforts.

Actually the space shuttle is the safest launch vehicle to date. From the ones that have at least 100 launches (in order to be able to properly compute stats for them), here are their failure rates as taken from http://www.ontonix.com/Blog/Outliers_-_understanding_Nature_... :

2% US Space Shuttle

5% R-7 (Russian Soyuz)

5% Ariane 1-4 (European)

6% Tsyklon (Russian)

7% Kosmos (Russian)

10% Thor/Delta/N1/N2/H1 (US)

11% Titan 2/3/4 (US)

12% Proton (Russian)

13% Kosmos 2 (Russian R-12)

14% Atlas (US)

Re: How We Nearly Lost the Discovery Shuttle

#14

Post by danso » Fri, Aug 17, 2012, 8:09 PM UTC

A great post, especially since it seeks to get at the truth of something that has implications for future missions, at the risk of the OP's reputation.

This part is one of the more disturbing parts though, and a good reminder of why technical persons of all fields, whether rocket scientists or programmers, should not adopt a "Well, we worked hard and we're smart so I'm sure everything's fixed"

> What you probably don’t know is that a side note in a final briefing before Discovery’s flight pointed out that the large chunk of foam that brought down Columbia could not have been liberated from an internal installation defect. Hmm. After 26 months of work, nobody knew how to address that little statement. Of course we had fixed everything. What else could there be? What else could we do? We were exhausted with study, test, redesign. We decided to fly.

How is it that this mentality exists at NASA? Isn't it a matter of logic that if the foam was shown not to have been an installation defect, that the engineers have to keep looking for the actual cause? The OP just brushes over this but surely there was some kind of debate, like: "Well, the particular test claiming that the foam was NOT an installation defect was poorly conducted, and all our other measurements say that the installation is the likely cause, so moving on..."

I really hope there isn't some kind of "Oh fuck it, just ship it" mentality at NASA.

Re: How We Nearly Lost the Discovery Shuttle

#15

Post by InclinedPlane » Fri, Aug 17, 2012, 8:12 PM UTC

vladd wrote: post #13

masklinn wrote: post #10

Earlier quoted context omitted.

In that, the Shuttle was very much like the Concorde: a unique and complex system beyond the edge of knowledge (at its creation), full of flaws and working through a combination of sheer luck and heroic efforts.

Actually the space shuttle is the safest launch vehicle to date. From the ones that have at least 100 launches (in order to be able to properly compute stats for them), here are their failure rates as taken from http://www.ontonix.com/Blog/Outliers_-_understanding_Nature_... : 2% US Space Shuttle 5% R-7 (Russian Soyuz) 5% Ariane 1-4 (European) 6% Tsyklon (Russian) 7% Kosmos (Russian) 10% Thor/Delta/N1/N2/H1 (US) 11%…

Those figures are a bit misleading because they include a lot of the early development phase of a vehicle in the operational history. Is it fair to include, say, the safety record of the Model-T when considering the safety of a 2012 model Ford Focus?

Re: How We Nearly Lost the Discovery Shuttle

#16

Post by vladd » Fri, Aug 17, 2012, 8:17 PM UTC

InclinedPlane wrote: post #15

vladd wrote: post #13

Earlier quoted context omitted.

Actually the space shuttle is the safest launch vehicle to date. From the ones that have at least 100 launches (in order to be able to properly compute stats for them), here are their failure rates as taken from http://www.ontonix.com/Blog/Outliers_-_understanding_Nature_... : 2% US Space Shuttle 5% R-7 (Russian Soyuz) 5% Ariane 1-4 (European) 6% Tsyklon (Russian) 7% Kosmos (Russian) 10% Thor/Delta/N1/N2/H1 (US) 11%…

Those figures are a bit misleading because they include a lot of the early development phase of a vehicle in the operational history. Is it fair to include, say, the safety record of the Model-T when considering the safety of a 2012 model Ford Focus?

I've tried to find other stats but I couldn't easily find more recent numbers. 2% is the lowest fatality rate I found. There are some statements at http://en.wikipedia.org/wiki/List_of_spaceflight-related_acc... but they're not changing the picture by much:

>> About five percent of the people that have been launched have died doing so. [..] About two percent of the manned launch/reentry attempts have killed their crew, with Soyuz and the Shuttle having almost the same death percentage rates.

Re: How We Nearly Lost the Discovery Shuttle

#17

Post by ak217 » Fri, Aug 17, 2012, 8:23 PM UTC

jevinskie wrote: post #2

"We informed the foam technicians at our plant in Michoud Louisiana that they were the cause of the loss of Columbia..." That to me is pretty disgusting. In an incident like the loss of Columbia, there is no one, true "root cause". To assign blame to those foam technicians was disingenuous and just another instance of "passing the buck" that seems to happen so often in the post-mortem of NASA failures. NASA knew of e…

To me, it's pretty refreshing. Leadership and responsibility should certainly come from the top and any public statement about this should make it clear that the leaders take responsiblity. Direct and clear communication about all components that you think contributed to the failure is just as critical. Saying "we failed as an organization" is not helpful, and saying "I failed as a leader" is nice but not informative. Failures can have root and contributing causes; organizations may have many shortcomings; calling them out in order of importance (as best you can determine it) and emphasizing what must be fixed is the only way to remove a poisonous kind of uncertainty that lingers in a situation like this.

Re: How We Nearly Lost the Discovery Shuttle

#18

Post by InclinedPlane » Fri, Aug 17, 2012, 8:24 PM UTC

vladd wrote: post #16

InclinedPlane wrote: post #15

Earlier quoted context omitted.

Those figures are a bit misleading because they include a lot of the early development phase of a vehicle in the operational history. Is it fair to include, say, the safety record of the Model-T when considering the safety of a 2012 model Ford Focus?

I've tried to find other stats but I couldn't easily find more recent numbers. 2% is the lowest fatality rate I found. There are some statements at http://en.wikipedia.org/wiki/List_of_spaceflight-related_acc... but they're not changing the picture by much: >> About five percent of the people that have been launched have died doing so. [..] About two percent of the manned launch/reentry attempts have killed their cre…

100 is a bit of an artificial number. The only launch vehicle in history that has had 100 manned launches has been the Shuttle, but that was a factor more of the wealth of the US than the inherent reliability of the system. Consider that Russia/USSR have only had about 2/3 of 100 total manned flights of any kind on any launcher.

Re: How We Nearly Lost the Discovery Shuttle

#19

Post by HarryHirsch » Fri, Aug 17, 2012, 8:29 PM UTC

jevinskie wrote: post #2

"We informed the foam technicians at our plant in Michoud Louisiana that they were the cause of the loss of Columbia..." That to me is pretty disgusting. In an incident like the loss of Columbia, there is no one, true "root cause". To assign blame to those foam technicians was disingenuous and just another instance of "passing the buck" that seems to happen so often in the post-mortem of NASA failures. NASA knew of e…

> "We informed the foam technicians at our plant in Michoud Louisiana that they were the cause of the loss of Columbia..."

This is stupid, perhaps wilfully so. The fact that foam can come off the external tank and strike critical parts of the shuttle is a fatal design flaw. Insulation has been used on rockets for sixty years now, and it has been observed to come loose every now an then. The difference is this: on a rocket it just falls off, but if any comes off the tank and strikes the Shuttle orbiter, there is disaster.

The fault was at an early stage of design.

Re: How We Nearly Lost the Discovery Shuttle

#20

Post by HCIdivision17 » Fri, Aug 17, 2012, 8:39 PM UTC

danso wrote: post #14

A great post, especially since it seeks to get at the truth of something that has implications for future missions, at the risk of the OP's reputation. This part is one of the more disturbing parts though, and a good reminder of why technical persons of all fields, whether rocket scientists or programmers, should not adopt a "Well, we worked hard and we're smart so I'm sure everything's fixed" > What you probably don…

It's more of a footnote problem. The critical details need to percolate up to public focus, and if that detail was only mentioned as a side note at the end, then that process is obviously failing. The scary bit is that the Challenger investigation showed this was a _huge_ issue, and apparently still existed many years later [1].

Granted, it's a terribly hard thing to fix, getting the right information to the right people with the right priority. But this shows how critical it is to do just that.

[1] Obligatory Tufte comments: http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0... (See also Feynman's comments on his experience on the investigation board in Surely You're Joking)