Boeing 737 MAX crash and the rejection of ridiculous data • Hacker News Board

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#21

Post by Ataraxic » Wed, Apr 10, 2019, 7:05 PM UTC

It should be noted in the doomed Air France 447 flight, the plane activated the stall warning because of a high angle of attack that was leading to stall. (thanks pdx for the corrected info)

At some point the system rejected the data and stopped the stall warning because the angle of attack was so severe that it considered the data erroneous. This is speculated to have caused the co-pilot to keep pulling back on the stick and to maintain the stall because everytime he let the nose of the plane come down the stall warning activated again as the AoA was decreasing into the range that the airplane considered a real signal. https://www.vanityfair.com/news/business/2014/10/air-france-...

A plane's instruments, the actions software takes, and it's interactions with the humans that fly the plane isn't as simple as an if statement.

If anything, fewer and simpler controls or automated systems are easier to debug and work around than a plane that has an internal calculus of what is valid data.

Air France 447 crashed because of a stall caused by a severe AoA that MCAS might have prevented (if MCAS simple pushes the nose down at high AoA it might. I am unsure of the exact implementation). MCAS obviously had a different impact on the Lion and Ethiopian Air Flights.

Basically, it's complicated.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#22

Post by area51 » Wed, Apr 10, 2019, 7:07 PM UTC

This needs more than an IF statement. This needs calculation of the rate of change of the variable, in addition to observing the value of the variable. Rate of change calculations are atleast an order of magnitude harder than just reading the variable. Doing it for infinite number of variables, in embedded, memory constrained, real-time systems is hard. So this wasn't going to be fixed via trivial 10 characters of code. But, yes, a more complex system would have avoided it.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#23

Post by phdp » Wed, Apr 10, 2019, 7:07 PM UTC

yongjik wrote: post #8

With all due respect, this sounds too much like the kind of armchair quarterbacking that routinely appears on HN when avionics/politics/astronomy is mentioned, where a lone programmer feels competent enough to criticize an industry for missing "something obvious". I mean, this particular change might have saved the particular 737, but I'd rather hear it from someone who actually knows how 737s fly.

Philip Greenspun is not a no one when it comes to aviation.

https://philip.greenspun.com/flying/resume

While he doesn't have 737 experience, he's far from an armchair quarterback.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#24

Post by astrodust » Wed, Apr 10, 2019, 7:07 PM UTC

gowld wrote: post #15

yongjik wrote: post #8

With all due respect, this sounds too much like the kind of armchair quarterbacking that routinely appears on HN when avionics/politics/astronomy is mentioned, where a lone programmer feels competent enough to criticize an industry for missing "something obvious". I mean, this particular change might have saved the particular 737, but I'd rather hear it from someone who actually knows how 737s fly.

Do you think the analysis is invalid? Do you think that an autopilot response to "AOA > 15 degrees" should be the same regardless of whether AOA is 20, 40, 60, or 80 degrees?

Is there an industrial coding standard that makes recommendations here?

What does NASA do?

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#25

Post by hw » Wed, Apr 10, 2019, 7:07 PM UTC

cpplinuxdude wrote: post #5

starpilot wrote: post #3

Now, imagine you have 5,000 such checks in millions of lines of flight control system code, many of them interdependent, and you have to fly the ship to test each one. You need to schedule time with the test pilots (who have lives of their own) and get the data dump from IT post-flight. It's aerospace so all this undergoes review, documentation, and signoff, and it all takes time. How do you prevent a single check fr…

I'm sorry i'll have to mention that the software can be thoroughly tested in simulation flights. Funnily enough i was involved with some virtualisation software used to test booking systems for Airports. If you can virtualise a booking systems, trust me you can virtualise the on-board flights systems.

No doubt you can test on-board flight systems, but you can't really compare virtualizing a booking system with flight systems, which are much more complicated and delicate. Now if you worked at SpaceX and worked on testing their flight systems, then I'd be more inclined to trust you.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#26

Post by Diederich » Wed, Apr 10, 2019, 7:07 PM UTC

chupa-chups wrote: post #2

This video from "Mentour pilot" has been deleted (supposedly upon request from boeing): https://www.youtube.com/watch?v=EzgBft-79U8 You can see it here (european version of youtube): https://vimeo.com/329558134 Or more info here: https://news.ycombinator.com/item?id=19627525

Strongly suggest everyone check out the video. It's...kind of shocking.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#27

Post by chupa-chups » Wed, Apr 10, 2019, 7:08 PM UTC

pdx_flyer wrote: post #12

starpilot wrote: post #3

Now, imagine you have 5,000 such checks in millions of lines of flight control system code, many of them interdependent, and you have to fly the ship to test each one. You need to schedule time with the test pilots (who have lives of their own) and get the data dump from IT post-flight. It's aerospace so all this undergoes review, documentation, and signoff, and it all takes time. How do you prevent a single check fr…

It's more an issue of Boeing essentially signing off on their own certification as functionally the same type when it is a very different plane. The AOA calculation is also much more complicated than the article explains. There is a speed element, engine power, etc.

AoA is just that: angle of attack (https://en.wikipedia.org/wiki/Angle_of_attack)

The sensor simply measures the angle of the airflow with respect to the airframe.

In fact it is just a free-moving surface attached to the airframe.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#28

Post by ckastner » Wed, Apr 10, 2019, 7:08 PM UTC

I have no practical experience with sensors but for those gained from a few university courses.

In one course, we were re-assembling systems on a weekly basis, and the touch-screens we used would all return somewhat quirky offsets, so I had to calibrate the input after each re-assembly because I could never be sure which module I was working with. I learned to never really trust a sensor.

And this was an undergraduate course. I would have assumed that in safety-critical applications, in a mature industry, performing every possible check on a sensor reading would be the obvious thing to do.

Was this particular case just out of the ordinary, or is it really that uncommon to do that?

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#29

Post by quanticle » Wed, Apr 10, 2019, 7:09 PM UTC

On the other hand, one of the contributing factors to the Three Mile Island nuclear disaster was the fact that the temperature sensors were showing their maximum programmed value of 280 degrees Centigrade. The actual core temperature was far far higher, but because the engineers designing the reactor never thought of the meltdown scenario, they programmed the temperature gauge to cut off at 280C, rejecting higher readings as obviously erroneous. This (along with a water gauge malfunction) led the reactor operators to misdiagnose the fault. The operators thought the reactor was overfull with coolant and was at risk of overpressure, when in reality the coolant was draining away.

I wouldn't be so quick to dismiss "obviously incorrect" sensor data.

Re: Boeing 737 MAX crash and the rejection of ridiculous data

#30

Post by tbabb » Wed, Apr 10, 2019, 7:10 PM UTC

There are several inexcusably egregious errors in the design of the MCAS system, and this "solution" addresses none of them.

- Single point of failure: The system makes command decisions based on the readings from a single sensor. The fact that nobody asked (or was bothered by the answer to) the question "what happens when that sensor fails?" is negligence.

- No re-training of pilots: Pilots were not aware of new ways in which the plane might take command away from them, and were left in the dark with only seconds to react to a deadly situation. The decision not to train was a cost and marketing motivated decision that sacrificed safety, to the tune of hundreds of lives lost.

Slapping on heuristics to condition unreliable data is not a good solution for life-critical systems. As another commentor pointed out, this is armchair quarterbacking, and it is not good armchair quarterbacking. This article should not be here.