How the Boeing 737 Max disaster looks to a software Developer • Hacker News Board

Re: How the Boeing 737 Max disaster looks to a software Developer

#121

Post by blunte » Thu, Apr 18, 2019, 10:36 PM UTC

alacombe wrote: post #76

andbberger wrote: post #67

Earlier quoted context omitted.

[citation needed] idk man, my code is pretty damn clean

Code cleanliness has no correlation with lack of bugs. Also, 1) your sample size is pretty small, and 2) what looks clean to you might very well be despicable to someone else.

And more importantly in this case, the appearance of your code is relatively meaningless compared to you failing to consider all cases of inputs.

Honestly, the appearance of code is irrelevant. The goal is to absolutely accurately meet the requirements - the true, actual requirements. It may not be within scope to know all the information you need to know to cover all the cases, and that's certainly a problem. But the cleanliness of your code, assuming it functions correctly, is unimportant to the operation. (It certainly will be a maintenance cost, though.)

Re: How the Boeing 737 Max disaster looks to a software Developer

#122

Post by blunte » Thu, Apr 18, 2019, 10:36 PM UTC

rdiddly wrote: post #64

rkagerer wrote: post #49

I believe the relative ease — not to mention the lack of tangible cost — of software updates has created a cultural laziness within the software engineering community. -- This --^ As someone who carefully crafts their code to strive for perfection, seeing sloppy work out there in the wild drives me nuts. I know folks here will deride me for being "inefficient", but in the long term I still maintain from my experience…

The most efficient way to do it is to do it once.

Indeed, but the prereq is that you know the need absolutely and correctly initially. And you usually don't.

Re: How the Boeing 737 Max disaster looks to a software Developer

#123

Post by stevespang » Thu, Apr 18, 2019, 10:37 PM UTC

EXECUTIVE SUMMARY: So Boeing produced a dynamically unstable airframe, the 737 Max. That is big strike No. 1. Boeing then tried to mask the 737’s dynamic instability with a software system. Big strike No. 2. Finally, the software relied on systems known for their propensity to fail (angle-of-attack indicators) and did not appear to include even rudimentary provisions to cross-check the outputs of the angle-of-attack sensor against other sensors, or even the other angle-of-attack sensor. Big strike No. 3.

Re: How the Boeing 737 Max disaster looks to a software Developer

#124

Post by Raidion » Thu, Apr 18, 2019, 10:37 PM UTC

anbop wrote: post #113

rkagerer wrote: post #49

I believe the relative ease — not to mention the lack of tangible cost — of software updates has created a cultural laziness within the software engineering community. -- This --^ As someone who carefully crafts their code to strive for perfection, seeing sloppy work out there in the wild drives me nuts. I know folks here will deride me for being "inefficient", but in the long term I still maintain from my experience…

Well, great code is inefficient if the impact of bad code is minimal. If you are shipping an animated emoji you should be sloppier with your code than if you are building flight control software. Overengineering software is like over-specing building materials by 10x and spending way more money on a building than it needs.

I agree, but practically a lot of bad software practices happen when middle management don't fully communicate the tradeoffs that are made when designing and building software. I find it hard to believe that this was a problem that no one really saw coming, but just didn't get flagged at a level that was needed to investigate it.

Re: How the Boeing 737 Max disaster looks to a software Developer

#125

Post by Sharlin » Thu, Apr 18, 2019, 10:37 PM UTC

_bxg1 wrote: post #63

"Neither such coders nor their managers are as in touch with the particular culture and mores of the aviation world as much as the people who are down on the factory floor, riveting wings on, designing control yokes, and fitting landing gears. Those people have decades of institutional memory about what has worked in the past and what has not worked. Software people do not." I wonder if, once basic programming become…

> What if the domain experts could not only speak the language of code, but could do some of the coding themselves?

Some can, especially in the academia. It… sort of works but is certainly not optimal. The truth is, it is not enough to be a coder who knows something about the domain, and neither is it enough to be a domain expert who can code. You have to be an expert at both, like the author of TFA. Or at the very least, your team has to be.

Re: How the Boeing 737 Max disaster looks to a software Developer

#126

Post by _bxg1 » Thu, Apr 18, 2019, 10:40 PM UTC

Sharlin wrote: post #125

_bxg1 wrote: post #63

"Neither such coders nor their managers are as in touch with the particular culture and mores of the aviation world as much as the people who are down on the factory floor, riveting wings on, designing control yokes, and fitting landing gears. Those people have decades of institutional memory about what has worked in the past and what has not worked. Software people do not." I wonder if, once basic programming become…

> What if the domain experts could not only speak the language of code, but could do some of the coding themselves? Some can, especially in the academia. It… sort of works but is certainly not optimal. The truth is, it is not enough to be a coder who knows something about the domain, and neither is it enough to be a domain expert who can code. You have to be an expert at both, like the author of TFA. Or at the very l…

Could there not be domain experts who know how to express their general ideas in terms of code (pseudocode, even), paired with coding experts who can architect the overall software system? You'd still need software experts, but you wouldn't have just software experts.

Re: How the Boeing 737 Max disaster looks to a software Developer

#127

Post by ncmncm » Thu, Apr 18, 2019, 10:40 PM UTC

d-sc wrote: post #112

ncmncm wrote: post #74

This is a very good analysis, but fatally incomplete. One really essential reason those planes crashed was that each time the MCAS triggered, it acted like it was the first time. If it added 1 degree of trim last time, it adds a second this time, a third next time, up to the five degrees that runs the trim all the way to the stops. A second reason is that, under the design still on file at the FAA, it could only add…

Architecting solutions is hard. In this case, you need knowledge of motors, flight controls, sensor fusion, etc. It’s easy to find edge cases when they present themself (tragically here). But most electrical-mechanical-software assemblies have similar issues.

Thus, the standards and procedures that if followed would have prevented the problem.

There is a standard that says automated controls are not allowed "authority" such that the pilot cannot counteract it.

There is a standard that says failure of a single sensor must not cause a critical failure.

Either violation alone makes the design not airworthy, and not certifiable for civil aviation, preventing both crashes.

Re: How the Boeing 737 Max disaster looks to a software Developer

#128

Post by inflatableDodo » Thu, Apr 18, 2019, 10:43 PM UTC

ken wrote: post #77

WalterBright wrote: post #65

Earlier quoted context omitted.

You'll see it in any piece of software written by any developer who is self-managed. The notion that software quality would greatly improve without managers is a myth.

We seem to have very different experiences. Every manager I ever had (including one at Boeing) has pushed me to ship software that wasn't at all ready, on the basis that the UI of the prototype looked good.

I am terrified these days of showing anything that looks like something that works to a project manager or client, without covering it in warning graphics.

Even photoshop mockups of UIs, introduced as, "Here's a photoshop mockup of the UI, what do you think?" will get some people to demand that it be shipped immediately.

Re: How the Boeing 737 Max disaster looks to a software Developer

#129

Post by Frost1x » Thu, Apr 18, 2019, 10:43 PM UTC

sisu2019 wrote: post #98

Sorry, I don't believe that our perspective on this is especially valuable or insightful. Everyone with an average IQ can explain in 5 sentences what went wrong with this plane. Yes new engines didn't fit the plane, fixed with software, well done. But why? It's not profits. Boeing has been for profit from the start. It's something more. We are getting bad at doing hard things. Here in Germany we can't seem to finish…

From my experience I disagree. It's not that we can't do hard things, it's that there's frequently more profitable approaches than taking difficult challenges. This is becoming systemic across our culture as ideas drawn from capitalism begin invading all aspects of our lives (including how we interact in personal relationships), not just business.

Pushing quicker time to market and cost saving solutions to the brink is part of the driving optimization problem businesses aim to do.

This search is done across various contributing costs but inevitably, safety becomes one of factors on the board. I have previous experience with the coal mining industry and I can attest that safety is frequently hit in the name of cost savings.

When you think about it, you rarely converge on an optimization solution across a search space from one direction, you typically have to cross boundaries to find the an optimum, especially for highly complex problems with many variables. Inevitably in this search, business interests will conflict with their profit motives as costs are cut down tighter and tighter in convergence. If we're lucky, it's a local optimum and some new technology or finding makes cost savings possible outside of cutting safety.

Re: How the Boeing 737 Max disaster looks to a software Developer

#130

Post by philjohn » Thu, Apr 18, 2019, 10:43 PM UTC

amelius wrote: post #89

rkagerer wrote: post #49

I believe the relative ease — not to mention the lack of tangible cost — of software updates has created a cultural laziness within the software engineering community. -- This --^ As someone who carefully crafts their code to strive for perfection, seeing sloppy work out there in the wild drives me nuts. I know folks here will deride me for being "inefficient", but in the long term I still maintain from my experience…

> As someone who carefully crafts their code to strive for perfection, seeing sloppy work out there in the wild drives me nuts. Do you use formal methods to prove the correctness of your code? Because that is what Boeing engineers do (I hope).

You would indeed hope. But then again, look at the Toyota ECU issues ... and the subsequent code review as part of the court case ...