Live data from Hacker News

How NASA built Artemis II’s fault-tolerant computer

cacm.acm.org

61–70 of 257 posts

Re: How NASA built Artemis II’s fault-tolerant computer

#61

>Effectively, eight CPUs run the flight software in parallel. The engineering philosophy hinges on a >“fail-silent” design. The self-checking pairs ensure that if a CPU performs an erroneous calculation >due to a radiation event, the error is detected immediately and the system responds. >“A faulty computer will fail silent, rather than transmit the ‘wrong answer,’” Uitenbroek explained. >This approach simplifies the…

In the Shuttle they would use command averaging. All four computers would get access to an actuator which would tie into a manifold which delivered power to the flight control surface. If one disagreed then you'd get 25% less command authority to that element.

Re: How NASA built Artemis II’s fault-tolerant computer

#62

>Effectively, eight CPUs run the flight software in parallel. The engineering philosophy hinges on a >“fail-silent” design. The self-checking pairs ensure that if a CPU performs an erroneous calculation >due to a radiation event, the error is detected immediately and the system responds. >“A faulty computer will fail silent, rather than transmit the ‘wrong answer,’” Uitenbroek explained. >This approach simplifies the…

These CPUs are typically implemented as lockstep pairs on the same die. In a lockstep architecture, both CPUs execute the same operations simultaneously and their outputs are continuously compared. As a result, the failure rate associated with an undetected erroneous calculation is significantly lower than the FIT rate of an individual CPU.

Put another way, the FIT (Failure in Time) value for the condition in which both CPUs in a lockstep pair perform the same erroneous calculation and still produce matching results is extremely small. That is why we selected and accepted this lockstep CPU design

Re: How NASA built Artemis II’s fault-tolerant computer

#63
post #55

I sure wish they would talk about the hardware. I spent a few years developing a radiation hardened fault tolerant computer back in the day. Adding redundancy at multiple levels was the usual solution. But there is another clever check on transient errors during process execution that we implemented that didn't involve any redundancy. Doesn't seem like they did anything like that. But can't tell since they don't ment…

One of the things I loved about the Shuttle is that all five computers were mounted not only in different locations but in different orientations in the shuttle. Providing some additional hardening against radiation by providing different cross sections to any incident event.

Re: How NASA built Artemis II’s fault-tolerant computer

#65
post #16

NASA didn't build this, Lockheed Martin and their subcontractors did. Articles and headlines like this make people think that NASA does a lot more than they actually do. This is like a CEO claiming credit for everything a company does.

Eh, in these kinds of subcontractor relationships there is a lot of work and communication on both sides of the table.

Re: How NASA built Artemis II’s fault-tolerant computer

#66
post #3

The quote from the CMU guy about modern Agile and DevOps approaches challenging architectural discipline is a nice way of saying most of us have completely forgotten how to build deterministic systems. Time-triggered Ethernet with strict frame scheduling feels like it's from a parallel universe compared to how we ship software now.

During the time of the first Apollo missions, a dominant portion of computing research was funded by the defense department and related arms of government, making this type of deterministic and WCET (worst case execution time) a dominant computing paradigm. Now that we have a huge free market for things like online shopping and social media, this is a bit of a neglected field and suffers from poor investment and mindshare, but I think it's still a fascinating field with some really interesting algorithms -- check out the work of Frank Mueller or Johann Blieberger.

Re: How NASA built Artemis II’s fault-tolerant computer

#68
post #57

Earlier quoted context omitted.

Agile is not meant to make solid, robust products. It’s so you can make product fragments/iterations quickly, with okay quality and out to the customer asap to maximize profits.

“Agile” doesn’t mean that you release the first iteration, it’s just a methodology that emphasizes short iteration loops. You can definitely develop reliable real-time systems with Agile.

> “Agile” doesn’t mean that you release the first iteration

Someone needs to inform the management of the last three companies I worked for about this.

Re: How NASA built Artemis II’s fault-tolerant computer

#69

Typo in the first sentence of the first paragraph is oddly comforting since AI wouldn't make such a typo, heh. Typo in the first sentence of the second paragraph is sad though. C'mon, proofread a little.

I think everyone should now make mistakes so we ca distinguish human vs ai.

Re: How NASA built Artemis II’s fault-tolerant computer

#70
post #3

The quote from the CMU guy about modern Agile and DevOps approaches challenging architectural discipline is a nice way of saying most of us have completely forgotten how to build deterministic systems. Time-triggered Ethernet with strict frame scheduling feels like it's from a parallel universe compared to how we ship software now.

Tesla’s Cybertruck uses that in its ethernet as well!

All the ADAS automotive systems use this, there are several startups in this space as well, such as Ethernovia.
Post reply on HN