Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Airbus A320 – intense solar radiation may corrupt data critical for flight

airbus.com

Post reply on HN
Print view View original

151–160 of 184 posts

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#151

Post by saltcured »

lambdaone wrote: post #103
RealityVoid wrote: post #98

Earlier quoted context omitted.

> they were given the specs and not allowed to communicate with the other teams in any way during the hw and sw development. Jeez, it would drive me _up the wall_. Let's say I could somewhat justify the security concerns, but this seems like it severely hampers the ability to design the system. And it seems like a safety concern.

What you are trying to minimize here is the error rate of the composite system, not the error rate of the individual modules. You take it as a given that all the teams are doing their human best to eliminate mistakes from their design. The idea of this is to make it likely that the mistakes that remain are different mistakes from those made by the other teams. Providing errors are independent , it's better to have th…

This seems like it would need some referees who watch over the teams and intrude with, "no, that method is already claimed by the other team, do something else"!

Otherwise, I can easily see teams doing parallel construction of the same techniques. So many developments seem to happen like this, due to everyone being primed by the same socio-technical environment...

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#152

Post by anonymousiam »

simne wrote: post #121
rkagerer wrote: post #52

Earlier quoted context omitted.

In redundant systems like these, how do you avoid the voting circuit becoming a single point of failure? Eg. I could understand if each subsystem had its own actuators and they were designed so any 3 could aerodynamically override the other 2, but I don't think that's how it works in practice.

> how do you avoid the voting circuit becoming a single point of failure They do not. Just make voting circuit much more reliable than computing blocks. As example, computing block could be CMOS, but voting circuit made from discrete components, which are just too large to be sensitive to particles. Unfortunately, discrete components are sensitive to overall exposure (more than nm scale transistors), because large sq…

Electronics in high-radiation environments benefit from a large feature size with regard to SEU reduction, but you're correct that the larger parts degrade faster in such environments, so they've created "rad-hard" components to mitigate that issue.

https://en.wikipedia.org/wiki/Radiation_hardening

It's interesting to me that triple-voting wasn't as necessary on the older (rad-hard) processors. Every foundry in the world is steering toward CPUs with smaller and smaller feature sizes, because they are faster and consume less power, but the (very small) market for space-based processors wants large feature sizes. Because those aren't available anymore, TMR is the work-around.

https://en.wikipedia.org/wiki/IBM_RAD6000

https://en.wikipedia.org/wiki/RAD750

Most modern space processing systems use a combination of rad-hard CPUs and TMR.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#155

Post by exe34 »

baq wrote: post #67
exe34 wrote: post #55

Earlier quoted context omitted.

if the issue is radiation bit flipping, you could make that part overly shielded?

Define ‘overly’. You can submerge it in a sphere of water, but that’s going to be expensive to launch.

I suspect a couple millimeters of lead in the right place would do it. cheaper to shield the voting mechanism than the whole thing.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#156

Post by oofbey »

This is in response to JetBlue flight 1230 from Cancun to Newark on October 30, 2025, where a cosmic ray of some kind flipped a bit and caused a dangerous situation. At the time there was a minor (G1) geomagnetic storm - meaning more cosmic rays than normal. The Planetary K-index was at 5. These are somewhat elevated numbers - enough to produce a visible Aurora in Canada, but probably not even the northernmost US. But also this level of space weather is also very common. We hit G1 or higher about once a week. That's the really damning part. If it had happened in a G4 or G5 storm, then the engineers might have responded "we can't fix everything", but this level of reliability is clearly unacceptable.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#157

Post by Loudergood »

jpollock wrote: post #148
jpollock wrote: post #146

The design of the system is very interesting, particularly how it expects to handle errors. In 90's Telco, you used to have a pair of systems and if they disagreed, they would decide which side was bad and disable it. In modern cloud, you accept there are errors. There's another request in ~10+ms. You only look when the error rate becomes commercially important. My understanding of spacecraft is that there would be 3…

For example.... "Preliminary A330/A340 FCPC algorithm" "The algorithm did not effectively manage a specific situation where AOA 2 and AOA 3 on one side of the aircraft were temporarily incorrect and AOA 1 on the other side of the aircraft was correct, resulting in ADR 1 being rejected." So, you've got a system where _two_ of the three sensors are bad, and you need to deal with it.

I'm in awe of the fact that two sensors can be wrong AND agree with each other.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#158

Post by Philip-J-Fry »

LadyCailin wrote: post #78
nickdothutton wrote: post #76

I’d just like to point out that if you are in the computing industry long enough, you will get to see a few such incidents under different circumstances, not only in industries like aerospace. Mostly things like ECC save your a*, sometimes your software will be able to recognise a temporary spurious reading and disregard it because you had enough alternative checking logic, or in the case of realtime and safety criti…

We noticed this in our logs once! We service a huge amount of traffic, and as part of that, we log what is effectively an enum. We did a summarization of this field once, and noticed that there were a couple of “impossible” values being logged. One of my coworkers realized that the string that actually got logged was exactly one bit off from a valid string, and we came to the conclusion that we were probably seeing c…

I also saw a similar thing. I also naively pointed at "cosmic rays". It wasn't until someone found the actual bug that I realised how unlikely that was.

The actual bug was unsafe code somewhere else in the application corrupting the memory. The application worked fine, but the log message strings were being slightly corrupted. Just a random letter here and there being something it shouldn't be.

The question really should have been, if this was truly cosmic interference, why only this service and why was the problem appearing more than once over multiple versions of the application?

Cosmic rays are a great excuse to problems you don't yet understand. But the reality of them is extremely rare and it's like 99% a memory corruption bug caused by application code.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#159

Post by russdill »

hughw wrote: post #150
russdill wrote: post #145

Earlier quoted context omitted.

No, because it's a completely different kind of radiation.

Different band, sure. Same principle, right?

The difference between ionizing and non-ionizing radiation is quite different. But for much of the radiation effecting electronics at high altitudes it's largely subatomic particles.

And of course you can block the type radiation that caused problems for the rpi with a good piece of paper.

Re: Airbus A320 – intense solar radiation may corrupt data critical for flight

#160

Post by rossjudson »

My armchair guess is that they had a new control pathway not properly participating in their integrity hand-off protocols, doing some kind of transformation outside of that protection.

I once saw some HW engineers go nuts trying to find out why a storage device had an error rate several orders of magnitude higher than the extremely low error rate they expected (and triggering data corruption errors). It turns out to be one extremely deep VHDL-based control area for an FPGA that didn't properly do integrity. You'd have to flip a bit at an incredibly precise point in time for error to occur, but that's what was happening. When all the math was said and done, that FPGA control path integrity miss exactly accounted for the the higher error rate.

Post reply on HN