Live data from Hacker News

How the Boeing 737 Max disaster looks to a software Developer

spectrum.ieee.org

201–210 of 306 posts

Re: How the Boeing 737 Max disaster looks to a software Developer

#201
post #183
post #156

Earlier quoted context omitted.

Ultimately it was not a technical failure: the failure was in allowing the plane to be sold with such a thoroughly bad design. Thus, a management and regulatory failure. Regulatory, because FAA signed off on it, obviously without applying any of the process that would have prevented it. Management, because the cost of this debacle will be many, many times what they saved by trying to skate by with a faulty design.

The airlines knew what they were doing when they didn’t pay for the “upgrade” and they doubly knew when they didn’t pay for it after the first crash. Airline management is just as culpable in this debacle.

There is more than enough blame to go around. E.g., Congress, for the last N decades, failing to fund FAA at levels clearly needed.

Re: How the Boeing 737 Max disaster looks to a software Developer

#202
post #162
post #157

Earlier quoted context omitted.

I don't know, I think the 3 points in the article make it glaringly obvious that the root cause is NOT engineering. The decisions made clearly ignored engineering and historical precedent at every turn. It's sad because Boeing has had some wonderful engineers, and Boeing aircraft have traditionally allowed the pilots to have the final say.

Of course, that much engineering failure is not possible without even more management and regulatory failure. But documenting the engineering failures is necessary to quantify the management failures, at least until discovery in the wrongful-death lawsuits begins.

I find that engineering vs management is not a very useful discussion point. Management is composed of engineers too and engineers are responsible for the output of their work. Production of something that doesn't work is an engineering failure.

It's depressing that everyone went along with this. Clearly there's a lack of impartial checks and balances in the process.

Re: How the Boeing 737 Max disaster looks to a software Developer

#203
post #107

I think a fundamental concept that is relatively ignored in press (mainstream or non-fringe at least) is in business priorities. As the essay points out, Boeing made decisions based on market (financial) factors all along the way. Of course they did - almost all companies do. Because it's so common, we forget it's not a RULE that cost, shareholder value, profit, etc. must be the final judgement. Certainly any company…

> Certainly any company that might prioritize safety (as in the case of a people-carrier) or some other non financial focus may not be as attractive to investors, in some cases - particularly where human life is concerned - maybe that's ok.

This reminds me of the Takata air bag recall. If your product is a life saving device, margins should not be the highest priority.

Re: How the Boeing 737 Max disaster looks to a software Developer

#204
Just a meta-comment, but I've noticed that for every single article posted about the 737 Max on HN, the top comments all tend to say that something in the linked article is dramatically wrong. I'm not sure what this means, but I find it interesting.

Re: How the Boeing 737 Max disaster looks to a software Developer

#205

Could someone please ELI5 what exactly went wrong that caused the crash? I read the article but I still don't understand if the crash was caused by (a) The MCAS doing something it wasn't programmed to do (by specification) (b) MCAS was working as expected, but the expectations/assumptions were wrong (excluding pilot mistakes). (c) MCAS worked as designed, and the design was correct IF the pilots behaved like Boeing e…

"All of the above"

The MCAS itself did exactly what it was programmed to do. One of the sensors feeding it, which was not redundant (!), malfunctioned, and MCAS acted on the data - garbage in, garbage out.

By not having sufficient redundancy, MCAS was designed in a failure-prone way. This was justified because the failure would be like another kind of known failure mode (runaway trim), and pilots already had a checklist for it (the key point being to turn off electric trim).

The problems with that are:

1. The problem a malfunctioning MCAS (acting "correctly" based on bad data) introduces is intermittent/erratic, making it harder to correctly react to it. Especially since the pilots weren't aware that a system that would act like this was on board!

2. Humans make mistakes in high-stress situations, which is why accepting something that introduces additional problems and relies on humans to handle them is a bad idea.

3. Turning off the electric trim also made it hard for the pilots to correct the situation, as it disables the manually-controlled but motor-powered trim. One theory is that the forces may have been too great for the backup solution (trimming manually) to work, a situation that old (but not current) manuals addressed.

MCAS has a limit how much it will trim, but any trim input by the pilot resets it, and lets it move the trim by the limit amount again.

This is just a layman's interpretation:

From the flight data recorder, it seems like the Ethiopian ET302 pilots only partially corrected the wrong trim, then did turn trim off. After this, there was no noticeable adjustment in trim until the pilots turned electric trim back on to be able to use it. They used it to trim a little bit, and 5 seconds after their last trim input, MCAS triggered again and drove them into the ground. [1]

On the other hand, the Lion Air JT610 crew doesn't seem to have turned trim off, just corrected it each time MCAS trimmed down (only for MCAS to trim back down 5 seconds later) [2] - until they once only trimmed a little against it, so that they hadn't corrected what the previous MCAS trim had done, but unlocked it for another one. MCAS applied more trim, and that was it.

This could likely have been avoided by telling the pilots about MCAS and how it works, having a separate switch that kills MCAS but not electric trim, some sort of indication that MCAS is working, some sort of indication that the AoA sensor is broken, redundant sensors, and countless other things. The most likely reason why those weren't implemented seems (for me) that adding them would likely trigger the certification/training requirements that Boeing was trying to avoid.

[1] https://leehamnews.com/2019/04/05/bjorns-corner-et302-crash-... [2] https://static.seattletimes.com/wp-content/uploads/2018/11/L...

Re: How the Boeing 737 Max disaster looks to a software Developer

#206
post #179
post #74

This is a very good analysis, but fatally incomplete. One really essential reason those planes crashed was that each time the MCAS triggered, it acted like it was the first time. If it added 1 degree of trim last time, it adds a second this time, a third next time, up to the five degrees that runs the trim all the way to the stops. A second reason is that, under the design still on file at the FAA, it could only add…

Stab trim cutout is not the only way to disable MCAS. Extending the flaps any amount also disables MCAS. Turning on autopilot also disables MCAS, though this isn’t entirely effective since spurious AoA readings may quickly disable the autopilot again.

Boeing put out an emergency airworthiness directive after Lion Air. It doesn't tell pilots to lower flaps. AoA sensor failure causes IAS Unreliable warnings, and the checklist for that item demands that flaps be left alone -- if you don't know how fast you're going, lowering flaps could cause a wing stall.

It's not reasonable to expect pilots to disobey checklists. We would all be less safe if they did. If pilots are following Boeing's instructions and planes are crashing, that's on Boeing.

Re: How the Boeing 737 Max disaster looks to a software Developer

#207
post #175

Earlier quoted context omitted.

> There isn't a visual indication for AOA Private pilot here. What you say is, strictly speaking, true but that doesn't mean that you can't tell an awful lot about what's going on by looking out the window, and, more importantly, at the other flight instruments. If your attitude, airspeed, and rate of climb are all looking normal, then if the AOA says you're stalling it's almost certainly wrong. And it it says that y…

>If your attitude, airspeed, and rate of climb are all looking normal, then if the AOA says you're stalling it's almost certainly wrong. Agreed - you can certainly sanity check the AOA data with other data sources. Unfortunately, I'm not sure that will always let you pinpoint AOA as the cause - imagine some nice dirt-loving wasps have built nests in all of your pitot tubes, or they've iced over, and now those values…

> imagine some nice dirt-loving wasps have built nests in all of your pitot tubes

All of them? And I didn't notice any of them during preflight? And none of them were there on the previous flight? And I didn't notice that the airspeed was not alive on the takeoff roll? Not going to happen.

> or they've iced over

Again, pretty freakin' unlikely on takeoff flying out of Addis Ababa. And this is another thing a pilot can rule out by looking out of the window. If you're not in clouds, you're not picking up ice.

Also, there are OAT (outside air temperature) sensors.

> your engines to have failed

Another thing that would be pretty apparent to the pilots.

No matter how you slice it, Boeing screwed the pooch bigly.

Re: How the Boeing 737 Max disaster looks to a software Developer

#208
post #151
post #74

This is a very good analysis, but fatally incomplete. One really essential reason those planes crashed was that each time the MCAS triggered, it acted like it was the first time. If it added 1 degree of trim last time, it adds a second this time, a third next time, up to the five degrees that runs the trim all the way to the stops. A second reason is that, under the design still on file at the FAA, it could only add…

Regarding #6...Boeing charged $80k to unlock the software feature to gather data from more than 1 angle of attack sensor.

The amount of misinformation on this point is astounding. That's not true in any way.

Re: How the Boeing 737 Max disaster looks to a software Developer

#209
post #196

Earlier quoted context omitted.

The article states that MCAS was implemented "on the hush-hush," which makes me wonder if it could even be subject to the same level of quality control as other features of the software. It might have had to bypass some of the more stringent parts of Boeing's development process to avoid appearing in documentation that the customer or the FAA might see.

This appears to be what happpened. If so, it amounts to criminal negligence. People should go to jail, but if anybody does, it will certainly not be the ones ultimately responsible. Most likely Boeing will pay fines and court judgments, something probably already factored into their stock price, impacting people holding the stock this year, not those who might have demanded better management five years ago. Certainly…

How about the fact that one plane full of people going down was not enough to wake everyone up. No, we need two planes to go down, and still the FAA was telling us we have nothing to be concerned about. The biggest problem here is not the engineering or even the management. Its regulatory capture.

Re: How the Boeing 737 Max disaster looks to a software Developer

#210
post #159
post #151

Earlier quoted context omitted.

Regarding #6...Boeing charged $80k to unlock the software feature to gather data from more than 1 angle of attack sensor.

In court that will cost them dearly: it demonstrates they were aware using two sensors would be safer. Probably AA or SWA demanded it, and Boeing compromised by charging for it. But... #6 was about paying attention to the other computer, not the other sensor. That would involve some big and expensive changes to the flight computer software, which they probably should have done long before the MAX project started. Mor…

It's nonsense. There was no paid option that affected the use of the second sensor.
Post reply on HN