71–77 of 77 posts
I saw this headline and my first thought was that someone was claiming that a mind impacting virus that evolved in the ocean was causing scientists to do research with less ambition. Which is of course ridiculous lol. But a bug in a visualization library impacting science is also ridiculous.
Earlier quoted context omitted.
Take a look at the linked chart in my other comment. Visualization is absolutely a driver during research, it isn’t just an embarrassing revelation. Charts killed the Challenger crew.
It can be a driver. But then you do a deep dive. You create frequency tables, you create crosstabs, you calculate summary statistics, you do inferential statistics. There is no excuse for not catching this pre-publication.
Earlier quoted context omitted.
I composed this comment concurrently with yours, so I'm moving it here as a response... This is going off-topic, but Tufte's attempt to cast the problem as fundamentally one of poor data presentation is rather self-servingly tendentious, IMHO, in a way that unfairly attributes a degree of culpability to the engineers who tried to stop the launch. The excellent video you link to, taken as a whole, supports this view,…
That’s an interesting point (and quite on topic; communication is key in STEM, and this ties in with that). Hypothetically, what would be the most fair argument in that situation? It’s quite remarkable that a line engineer was convinced the rocket was going to explode, even to the extent of hopping in his car with his daughter and frying to stop the launch after his company gave the go-ahead. Data presentation seems…
In Tufte's version, the meeting in question was the tipping point where it all went wrong, while the reality is that it was the last forlorn chance for NASA to to escape, by the skin of its collective teeth, from an overdue disaster that had been years in the making. As the Rogers Commission revealed, NASA had, in an environment of over-promising and political horse-trading, developed a culture in which deviance was normalized, and it was not ready to handle evidence contrary to the semi-official dogma of shuttle flights being routine and established events.
I'm not in a position to say how Boisjoly felt so sure the launch would end disastrously, but I can make a few guesses. I think it is quite possible that he gradually became aware, and then concerned, that the O-rings did not fare well in cold weather, as the data trickled in one launch at a time. I can imagine that when it became clear, a few days before the launch, that the temperature would be below freezing, his concerns sharpened into near-certainty that things would go wrong. One does not need a theory of what, precisely, was happening to the O-rings to suppose that if below-normal temperatures led to problems, then nothing good could come from an exceptionally low one. Perhaps he was too close to the data; I can also imagine that this seemed so clear to him that he never imagined his managers - who were also engineers - not also seeing it, instead clinging to older estimates of risk. I further imagine that he was completely blindsided by the somewhat rhetorical and sarcastic response, which went something like "are we supposed to wait until July?"
IIRC, Boisjoly anticipated that the joints would fail catastrophically immediately after the boosters were ignited, and for a minute afterwards he experienced profound relief...
Despite this all coming out in the Rodgers Commission report, NASA followed the same normalization of deviance path after it became apparent that ice was damaging the tiles, which is one reason why I doubt that better charts would have stopped the launch.
There are (at the time of posting this comment) no comments raising any substantive issue with the arxiv submission itself (which ofc has to go through the peer review process of publication, and hopefully the original authors will respond / rebut this new article) - so curious why its been flagged? It’s not dead, so cannot vouch for it.
If folks in the HN community who have flagged it have done so because there are serious issues with what the paper is asserting, please comment / critique instead of just flagging it. If it’s because of the ambiguity in the title, I hope @dang and the moderators editorialize - there are some valuable comments in this thread that helped me understand what the issue is and what the bug is!
Earlier quoted context omitted.
That’s an interesting point (and quite on topic; communication is key in STEM, and this ties in with that). Hypothetically, what would be the most fair argument in that situation? It’s quite remarkable that a line engineer was convinced the rocket was going to explode, even to the extent of hopping in his car with his daughter and frying to stop the launch after his company gave the go-ahead. Data presentation seems…
My position is not that a different or more thorough presentation of the argument would have made no difference (though, personally, I doubt that it would have), it is that Tufte's argument (and the aphorism it launched, "charts killed the Challenger crew") greatly exaggerates the significance this narrow issue and tacitly blames the people who were trying their best to save the day. In Tufte's version, the meeting i…
Earlier quoted context omitted.
My position is not that a different or more thorough presentation of the argument would have made no difference (though, personally, I doubt that it would have), it is that Tufte's argument (and the aphorism it launched, "charts killed the Challenger crew") greatly exaggerates the significance this narrow issue and tacitly blames the people who were trying their best to save the day. In Tufte's version, the meeting i…
(Just wanted to say thank you for the thoughtful followup. I was hoping you would, since it sounded like it’d be interesting. And when you phrase it like you did, it does sound absurd to offload the blame onto whoever was presenting the charts. Enjoy the rest of your Sunday.)
https://waynehale.wordpress.com/category/after-ten-years/
Also, I recall reading somewhere that the chairman of the Columbia Accident Investigation Board, Admiral Harold Gehman, decided to conduct a test to see if the piece of foam seen hitting the leading edge could have broken it. As, at the time, it had not been decided to end the shuttle program, this was not an easy decision, as it meant sacrificing an essentially irreplaceable spare part. What finally convinced Gehman to go ahead was the fact that a great many NASA engineers firmly believed it could not possibly have been the cause.
Earlier quoted context omitted.
No, the question is: how did peer review not catch it? I have the impression that reviewers don't have the time or incentive to give papers more that a cursory review. Independent of this case, a great many papers are published where the only "proof" is a user study or survey with an extremely low number of participants, but it still gets published. Many papers don't publish their datasets and don't contain enough de…
Can you describe how you would have expected peer reviewers to catch this?