311–320 of 361 posts
How is this a $500 million mistake? It seems the issues didn’t cost $500 million.
It's not "mistake" singular, it's actually mistakes. The first mistake is the $500 million rover is fried. The second mistake is believing the first mistake. (or put another way, the first mistake cost $500 million, the second mistake which they didn't realise at the time, saved $500 million) But you can't explain the second mistake without first explaining the first mistake, hence the title.
Earlier quoted context omitted.
Aerospace-grade connectors are specifically designed to support multiple keyings that prevent this kind of thing. It's definitely a problem preventable by careful design if the interface supports making this kind of mistake.
Can confirm. Source: I used to work for NASA, and I'm a private pilot. There are literally millions of electrical connections that get made on aircraft and spacecraft on a regular basis and I can't think of ever hearing of an incident caused by one of them being made backwards. (Now, mechanical connections getting made backwards is not unusual. That's why you check to make sure that the flight control surfaces move i…
How … how often does that go wrong?!
I agree that the person who made such a mistake will be the person who never makes that mistake again. That's why firing someone who has slipped up (in a technical way) and is clearly mortified is typically a bad move. However, I don't agree that this is the "real" lesson. Given the costs at play and the risk presented, the lesson is that if you have components that are tested with a big surge of power, give them cus…
I agree that the person who made such a mistake will be the person who never makes that mistake again. That's why firing someone who has slipped up (in a technical way) and is clearly mortified is typically a bad move. However, I don't agree that this is the "real" lesson. Given the costs at play and the risk presented, the lesson is that if you have components that are tested with a big surge of power, give them cus…
Well, it's true that it should be designed such that they cannot be plugged incorrectly. I would imagine it is indeed mostly designed in that way, but there can still be erroneous configurations that were not accounted for at the design stage. Especially during testing you're often dealing with custom cables connectors and circuits that are different from the "normal configuration". I would say that the lesson is to…
I agree with you, but on Earth this is easy. For spacecraft I imagine you can't just use any connector from Digikey
> especially with timeline pressure etc.
If timeline pressure, lost sleep, or rushing jobs not meant to be rushed causes a catastrophic technical error to be made, it is 100% the fault of the person who imposed the timeline, whether that be some middle manager, vice president, board, investor, or whoever. Emphatically NOT the engineer who did the work, if they do good work when not under time pressure.
HOLD PEOPLE LIABLE for rushing engineers and technicians to do jobs that require patience and time to do right.
Earlier quoted context omitted.
> Still, I cannot imagine why anyone thought it was an acceptable risk, to use a 100% oxygen atmosphere with Apollo 1 Especially when, in prior experience there, asbestos had caught fire in the same situation (O2, low pressure).
Wow, this detail I did not know yet. It just was a reckless rush to the moon, no matter the cost at this time. Without the deaths, nothing would have changed probably.
I agree that the person who made such a mistake will be the person who never makes that mistake again. That's why firing someone who has slipped up (in a technical way) and is clearly mortified is typically a bad move. However, I don't agree that this is the "real" lesson. Given the costs at play and the risk presented, the lesson is that if you have components that are tested with a big surge of power, give them cus…
Earlier quoted context omitted.
What strikes me as remarkable in all such stories is how almost always, the person committing the mistake is a junior who never deserves the blame. And how cavalier the handoff/onboarding by the 'seniors' working on the projects are. Having worked in enough of these though, I am aware that even they (the "seniors") are seldom entirely responsible for all the issues. It's mostly business constraints that forces cuttin…
As I said on Slack the other day in response to a similar story, "If, on your first day, you can destroy the prod database, it's not your fault." (One of my standard end-of-interview questions is "how easy is it for me to trash the production database?" Having done this previously[1] and had a few near misses, it's not something I want to do again.) [1] In my defence, I was young and didn't know that /tmp on Solaris…
I’ve had a search but can’t work out why it’s special.
Earlier quoted context omitted.
Our test suite expects that the database name has a `_test` suffix, so you can't run the tests even locally without the suffix.
Our test harness takes an optional template as input and immediately copies it. It’s useful to distribute the test anyway, especially for non-transactional tests. If the database initialisation is costly that’s useful even if tests run on empty, as copying a database from a template is much faster than creating one DDL by DDL, for postgres at least.
Are the electronics in these rovers really so bespoke that they don't have multiple copies of each electronic component warehoused on-site? I'd expect that the rover body itself would be bespoke this late in the process, although a parallel test vehicle would be useful, do they have that?). But in case someone fried the rover's electronics I'd think tearing it apart and replacing them while maintaining the chassis sh…
They almost certainly had flight spares but with two weeks until your launch window, there is zero chance you are deintegrating multiple systems, swapping in the spare, reintegrating, and re running your acceptance test campaigns. And that is assuming that they damaged a subsystem. Back powering the entire spacecraft could have wrecked your power system and anything connected to it. You'd have to disposition every pa…
According to Wikipedia they could have stretched those 2 weeks to around 3 weeks, but after that they'd have missed the launch window.
The usual processes are there to have a near-certainty of a working rover, but under these circumstances I'd think they'd just YOLO it and hope for the best.
But that assumes they've got spare electrical components, or alternatively a better use for the booster sitting on the pad than such an improvised mission.