Lack of redundancies on Boeing 737 MAX baffles some involved in developing it • Hacker News Board

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#131

Post by cjbprime » Wed, Apr 03, 2019, 3:53 PM UTC

alistairSH wrote: post #104

cjbprime wrote: post #83

Earlier quoted context omitted.

It doesn't present as runaway trim, though. It's a small change in the trim, repeated every few seconds, which can be counteracted on the control column but will eventually add up. Small trim changes are usually happening all the time.

This is my understanding. In a typical run-away trim, the trim wheel will move a great distance. It's very noticeable, both audibly (the wheel makes a clack-clack-clack noise) and visibly (there are white paint flashes on the wheel). And, obviously, the plane nose goes up or down by more than expected. The correct remedy is to disable the auto trim control via switches on the panel (located near the trim wheels). The…

Yeah. I'm also not clear on the practicality of retrimming the aircraft at low altitude (Ethiopian was never >1000ft above ground) before impacting terrain after you disable the trim motor. You have to use a hand crank, because you just killed the trim motor altogether, and I think it takes several minutes (!) of cranking to get the jackscrew from one end to the other.

I wonder if they killed the trim motor and then failed to reach level flight before the ground got in the way.

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#132

Post by wjnc » Wed, Apr 03, 2019, 4:04 PM UTC

hbarka wrote: post #123

crocal wrote: post #61

I am a designer and implementor of vital speed and distance measurement systems, with triply modular redundancy, used in mass transit application. I cannot even imagine what the designers of this thing are going through now. It must be terrible. To make it worse, it's a confusing topic. There are two pillars to the design of such system. 1. Faulty sensor must be detected with a very high probability. The typical way…

I enjoyed your comment because I’m particularly interested in Tesla’s Autopilot system and your mention of “graceful degradation and clean handover” sounds like it should apply to driving autonomy levels as well.

In the way we use cars there seems no time for graceful handover. You're about the same time interval away from the next car as the time to gain situational awareness, probably less. A plane is usually quite some minutes away from the ground and other planes, or f.e. when landing on autopilot the pilots are not distracted. If you'd apply this standard, the bar for autopilot in cars would be very high indeed. (That might be warranted, although in a risk / reward perspective humans are terrible drivers too.)

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#133

Post by magduf » Wed, Apr 03, 2019, 4:08 PM UTC

dingaling wrote: post #24

brokenmachine wrote: post #3

Earlier quoted context omitted.

They need triple temperature sensors on those sensors to sense when they freeze...

Although you're being facetious, AoA sensors do have self-diagnostics that determine when their heating elements have failed. Incodentally on the A380 and A350 Airbus have changed to combined AoA-pitot sensors of which there are now four.

Sounds like Airbus airplanes are much, much safer to fly on than Boeings. Why would anyone trust a company which thinks it's OK to have a flight control system that only uses input from a single sensor that's known to fail?

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#134

Post by ken » Wed, Apr 03, 2019, 4:09 PM UTC

> "A properly trained pilot should be able to solve an MCAS anomaly or any uncommanded flight-control input through procedures that are taught to all 737 pilots"

It blows my mind that anyone still talks like this.

My first job was writing (non-life-critical) scientific software for detail-obsessed university researchers working in a lab with notebooks and procedures. If "just solve it with training" could work for anyone in the world, it would have been these people. Yet it was immediately and abundantly clear that designing usability into the system would have far more impact on their success rate at any task than documentation or training or even the reliability of the software itself.

The movie "Apollo 13" showed astronauts working in a simulator, and engineers giving them faulty sensor inputs to try to trick them. They had to train to be able to identify this possibility, and react correctly to it. This isn't the sort of thing you can figure out on the fly (so to speak). If the solution to flying an aircraft with a faulty MCAS sensor was "the pilot should have read it in a book last month", it's no wonder they were in trouble.

The software industry does make some programs with random inputs and hidden internal state, and require users to figure out what's going on and solve it anyway. We call them "games". Perhaps the defining characteristic of a game is that, even if you've read the manual, you won't succeed on the first try.

> "But, he said, if he were designing the system from scratch, he would emphasize the training while also building the plane with three sensors."

Is even three enough? From an earlier version of the "2001" Wikipedia page [1]:

> "In the story HAL features a design with triple redundancy, so that if one of the three modules fails the other two can outvote it. However, there is a [[theorem]] in [[computer science]] that proves that for such [[distributed systems]] a vote-based [[sanity check]] only works if ''less than'' one third of the modules fail. Thus the failure of a single one of HAL's redundant modules would be sufficient to compromise the system, as apparently happened in the movie."

I don't know what theorem this is referring to. Help?

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#135

Post by magduf » Wed, Apr 03, 2019, 4:11 PM UTC

golergka wrote: post #88

iSnow wrote: post #85

Earlier quoted context omitted.

But management also relies on customers to tell them which compromise is vital to selling a product. And in a market with such huge price-pressure as aeronautics, I can easily see how this is going to override engineer's concerns.

I don't think that any Boeing customer have said or implied that it's OK if a plane can crash as long as they save up on pilot training.

Of course they didn't; the damage to the airline's reputation would be terrible. The customers didn't make this choice; they were deceived by Boeing, who sold them on the idea that they didn't need any significant new training for this disaster of an aircraft. The customers didn't design this thing, they just bought it after listening to Boeing's promises.

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#136

Post by magduf » Wed, Apr 03, 2019, 4:15 PM UTC

ishan1121 wrote: post #81

ptidhomme wrote: post #68

Earlier quoted context omitted.

Automation is not bad per se. What I mean is that there must be an opportunity for human override at all levels of automation (and as seamless as possible) in case of machine failure. It does not deny that it could be done by one pilot only, or half a pilot or even somehow a remote human intervention. When designing a critical system such as an aircraft, you must include human authority into it. No technology will ev…

>When designing a critical system such as an aircraft, you must include human authority into it This is where Boeing messed up big time. According to the article they only had 2 AoA sensor, completely missing the fact that what happens if one sensor fails and which is correct. Airbus has 3, but IMO a critical component like an AoA sensor there should be 5 + additional inputs from the artificial horizon should be cons…

>This is where Boeing messed up big time. According to the article they only had 2 AoA sensor, completely missing the fact that what happens if one sensor fails and which is correct.

No, you missed something really critical in all this. There are 2 AoA sensors, however only 1 of them is used as input for the MCAS system. There was no redundancy whatsoever!

The proposed software patch will use both AoA sensors, and light an indicator light if they disagree. They don't want to retrofit the planes with 3 sensors like Airbus, because obviously Boeing cares more about profits than safety and good engineering.

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#137

Post by magduf » Wed, Apr 03, 2019, 4:17 PM UTC

jussij wrote: post #66

From the information that keeps leaking out it appears the 737 MAX has some serious design faults. What is going to interesting is to see how quickly the FAA certifies the new software changes this second time around. The current perception would be the FAA was too quick to certifying the original MCAS as being safe. I suspect the second time around that certification process is going to take a lot more time and effo…

>What is going to interesting is to see how quickly the FAA certifies the new software changes this second time around.

With a Trump lackey running the FAA, I'm sure they'll rubber-stamp whatever Boeing puts out there.

What I want to see is if any of the foreign regulators refuse to certify it. I'd love to see a show-down in international aviation between the Trump administration and better-run nations.

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#138

Post by magduf » Wed, Apr 03, 2019, 4:19 PM UTC

hairytrog wrote: post #26

The only way to be 100% safe and 100% fool proof is to not fly. For a mechanical system as complex as an airplane, triple redundancy with a good pilot is pretty much the best you can hope for.

Triple redundancy makes planes extremely safe to fly. Airbus has triple redundancy, and Boeing doesn't believe in it because they care more about profits. So the simple answer is to avoid flying on Boeing aircraft, and stick to Airbus ones.

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#139

Post by golergka » Wed, Apr 03, 2019, 4:33 PM UTC

iSnow wrote: post #119

golergka wrote: post #95

Earlier quoted context omitted.

They certainly voted with their wallets for a plane that doesn't require additional training. But they sure as hell did not vote with their wallets for a plane that does that on expense of crashing. And, to reiterate my point, I think that managers who pushed for trade-offs between different objectives did not push for this particular trade-off either.

Well, that's the law of unintended consequences. Customers wanted so not pay for retraining, Boeing wanted to get to the market faster and not wait for re-certification, managers wanted a physical problem fixed in software - and in the end the envelope got pushed too far. I am sure the company operating the Titanic didn't exactly strive to have a shipwreck either.

This is true, but it is a statement about causality and the thread was about responsibility. Those are two different things. The fact that your actions lead to a certain outcome does not automatically mean that you're morally responsible for this outcome (that's the fallacy that leads to victim blaming, among other things).

It is always engineer's responsibility to clearly explain the trade-offs to the managers. Only a manager who's making the decision with accurate information can be responsible for it.

Re: Lack of redundancies on Boeing 737 MAX baffles some involved in developing it

#140

Post by linuxftw » Wed, Apr 03, 2019, 4:38 PM UTC

oldjokes wrote: post #98

crocal wrote: post #61

I am a designer and implementor of vital speed and distance measurement systems, with triply modular redundancy, used in mass transit application. I cannot even imagine what the designers of this thing are going through now. It must be terrible. To make it worse, it's a confusing topic. There are two pillars to the design of such system. 1. Faulty sensor must be detected with a very high probability. The typical way…

My car is only five years old but already has two sensors that have gone bad in the engine. Neither of them are vital and everything works great despite the occasional check engine light. Redundancy. It's a time intensive process to pop the head off and replace sensors. Nobody wants to pay their mechanic $100 an hour if they can avoid it. I can't even imagine a sensor failure having zero redundancy, much less changin…

Failed sensors can lead to premature failure of various components, especially catalytic converters: http://ricksfreeautorepairadvice.com/causes-catalytic-conver...

Don't put off a $150 job and cost yourself a $2k job. Most sensor replacements don't require removing the cylinder heads.