Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Airbus A320 – intense solar radiation may corrupt data critical for flight

airbus.com

Post reply on HN
Print view View original

51–60 of 184 posts

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#51

Post by f1shy »

CrossVR wrote: post #45
vel0city wrote: post #29

Earlier quoted context omitted.

I'm reminded of the Apollo moon landing where the computer was rapidly rebooting and being in an OK-ish state to continue to be useful almost immediately

It wasn't rebooting, it ran out of memory and started aborting lower priority tasks. It was a excellent example of robust programming in the face of unexpected usage scenarios.

Of topic for the thread, but on for the comment: I was working in an automotive project 3 years ago. It was all about safety, and one hypothesis was the processor could get overloaded. I was astonished no one in a grouo of 20 “senior sw architecs” had any idea about the concept of load shedding. The proposed solution was “in that case, reboot”.

Mind you whatever came out of that project is rolling on the street today.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#52

Post by rkagerer »

anonymousiam wrote: post #43
addaon wrote: post #7

I’d really, really like to know what microcontroller family this was found on. Assuming that this is a safety processor (lockstep, ECC, etc) it suggests that ECC was insufficient for the level of bit flips they’re seeing — and if the concern is data corruption, not unintended restart, it means it’s enough flips in one word to be undetectable. The environment they’re operating in isn’t that different from everyone els…

proper SEU mitigation goes far beyond ECC. Satellites fly higher than the A320, and they (at least the ones I know about) use Triple Modular Redundancy: https://en.wikipedia.org/wiki/Triple_modular_redundancy https://en.wikipedia.org/wiki/Single-event_upset For manned spaceflight, NASA ups N from 3 to 5. Other mitigations include completely disabling all CPU caches (with a big performance hit), and continuously refre…

In redundant systems like these, how do you avoid the voting circuit becoming a single point of failure?

Eg. I could understand if each subsystem had its own actuators and they were designed so any 3 could aerodynamically override the other 2, but I don't think that's how it works in practice.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#53

Post by f1shy »

joelthelion wrote: post #34

Do they really need to ground the entire fleet for that? One incident for ten thousand planes in the air for years. I'd think that giving airlines two months to fix it would be sufficient.

I would personally not want to seat in those planes in those 2 months.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#54

Post by AlphaSite »

rkagerer wrote: post #52
anonymousiam wrote: post #43

Earlier quoted context omitted.

proper SEU mitigation goes far beyond ECC. Satellites fly higher than the A320, and they (at least the ones I know about) use Triple Modular Redundancy: https://en.wikipedia.org/wiki/Triple_modular_redundancy https://en.wikipedia.org/wiki/Single-event_upset For manned spaceflight, NASA ups N from 3 to 5. Other mitigations include completely disabling all CPU caches (with a big performance hit), and continuously refre…

In redundant systems like these, how do you avoid the voting circuit becoming a single point of failure? Eg. I could understand if each subsystem had its own actuators and they were designed so any 3 could aerodynamically override the other 2, but I don't think that's how it works in practice.

Voting can be coordinated between the N cpus rather than an external arbiter (even making that redundant eventually required the CPUs to decide what to do if they disagree so may as well handle it internally).

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#55

Post by exe34 »

rkagerer wrote: post #52
anonymousiam wrote: post #43

Earlier quoted context omitted.

proper SEU mitigation goes far beyond ECC. Satellites fly higher than the A320, and they (at least the ones I know about) use Triple Modular Redundancy: https://en.wikipedia.org/wiki/Triple_modular_redundancy https://en.wikipedia.org/wiki/Single-event_upset For manned spaceflight, NASA ups N from 3 to 5. Other mitigations include completely disabling all CPU caches (with a big performance hit), and continuously refre…

In redundant systems like these, how do you avoid the voting circuit becoming a single point of failure? Eg. I could understand if each subsystem had its own actuators and they were designed so any 3 could aerodynamically override the other 2, but I don't think that's how it works in practice.

if the issue is radiation bit flipping, you could make that part overly shielded?

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#56

Post by Reason077 »

RealityVoid wrote: post #46
addaon wrote: post #7

I’d really, really like to know what microcontroller family this was found on. Assuming that this is a safety processor (lockstep, ECC, etc) it suggests that ECC was insufficient for the level of bit flips they’re seeing — and if the concern is data corruption, not unintended restart, it means it’s enough flips in one word to be undetectable. The environment they’re operating in isn’t that different from everyone els…

See my other comments in the other threads. This does not have EDAC. I was as surprised as you but it doesn't seems to be an MCU but a composition of several distinct chips. That flight computer was designed in the 90's and updated in 2002 with a new hw variant that does have edac. So yes, for this kind of thing, I can buy that a bit flip happened. You can see much more data in the report: https://www.atsb.gov.au/sit…

The recalled aircraft include the latest A320neo model, some of which are basically brand new. Why would they be using flight computers from before 2002? Why is an old report from 2008, relating to a completely different aircraft type (A330), relevant to the A320 issue today?

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#57

Post by RealityVoid »

Reason077 wrote: post #56
RealityVoid wrote: post #46

Earlier quoted context omitted.

See my other comments in the other threads. This does not have EDAC. I was as surprised as you but it doesn't seems to be an MCU but a composition of several distinct chips. That flight computer was designed in the 90's and updated in 2002 with a new hw variant that does have edac. So yes, for this kind of thing, I can buy that a bit flip happened. You can see much more data in the report: https://www.atsb.gov.au/sit…

The recalled aircraft include the latest A320neo model, some of which are basically brand new. Why would they be using flight computers from before 2002? Why is an old report from 2008, relating to a completely different aircraft type (A330), relevant to the A320 issue today?

Because the problem isn't just this. It's that the flight controller did not properly decide what to do when the data spiked because of this issue as well.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#58

Post by Havoc »

Reason077 wrote: post #56
RealityVoid wrote: post #46

Earlier quoted context omitted.

See my other comments in the other threads. This does not have EDAC. I was as surprised as you but it doesn't seems to be an MCU but a composition of several distinct chips. That flight computer was designed in the 90's and updated in 2002 with a new hw variant that does have edac. So yes, for this kind of thing, I can buy that a bit flip happened. You can see much more data in the report: https://www.atsb.gov.au/sit…

The recalled aircraft include the latest A320neo model, some of which are basically brand new. Why would they be using flight computers from before 2002? Why is an old report from 2008, relating to a completely different aircraft type (A330), relevant to the A320 issue today?

> Why would they be using flight computers from before 2002?

Guessing that using previously certified stuff is an advantage

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#59

Post by raverbashing »

Apparently the fix is reverting to a previous version of the SW (see https://avherald.com/h?article=52f1ffc3&opt=0 )

Curious what a sw change might have done in terms of resiliency. Maybe an incorrect memory setting or some code path that is not calculating things redundantly maybe?

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#60

Post by Havoc »

jfoster wrote: post #49
joelthelion wrote: post #34

Do they really need to ground the entire fleet for that? One incident for ten thousand planes in the air for years. I'd think that giving airlines two months to fix it would be sufficient.

I wonder who eats the cost of this? I presume it's the airlines. So the immediate cost to Airbus of grounding the fleet is quite low, whilst the downside of not grounding the fleet (risk of incident, lawsuits, reputation, etc.) could be substantial.

Yeah should be airlines

It sounds like the fix is fairly quick so probably not as expensive as the max multi month groundings

I doubt anyone is going to sue. Repairs etc are a part of life when owning aircraft. So as long as Airbus makes this happen fast and smooth they’re probably ok

Post reply on HN