Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Boeing's 737 Max software outsourced to lower-paid engineers

bloomberg.com

Post reply on HN
Print view View original

131–140 of 166 posts

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#131

Post by TarlanT »

ganeshkrishnan wrote: post #87
crorella wrote: post #11

Money saved by firing experienced engs and outsourcing the development Why is that so hard to understand? Massive fail at risk management maybe?

The software did exactly as it's supposed to do: If the AOA sensors tells you the plane is tilting too high up, pitch the nose down a little. The main issue was that there was only one AOA sensor connected in standard packages and the hardware for it failed. The pitot tubes failed for the doomed Air France incident which caused the software to read incorrectly and stall. For what it's worth AirBus has a huge engineer…

Experienced software engineer would NEVER code something like that. Both physically and logically it makes no sense to get same AOA reading after many iterations of trimming. That clearly means something is wrong. And probably the sanity of data being fed is in question. It’s the dumbest mistake in critical system I’ve ever heard. Even in non critical software these types of logical mistakes are rare.

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#132

Post by ganeshkrishnan »

dTal wrote: post #130
ganeshkrishnan wrote: post #87

Earlier quoted context omitted.

The software did exactly as it's supposed to do: If the AOA sensors tells you the plane is tilting too high up, pitch the nose down a little. The main issue was that there was only one AOA sensor connected in standard packages and the hardware for it failed. The pitot tubes failed for the doomed Air France incident which caused the software to read incorrectly and stall. For what it's worth AirBus has a huge engineer…

This comment is glib and misinformed. Firstly, no the software did not do exactly as it's supposed to. It crashed the plane by rendering it unflyable. It's not supposed to do that. And your description of the MCAS specification is vastly over-simplified - it is not an autopilot and its job is to gradually change the behaviour of the plane in different flight modes (which it must detect), not to directly fly the plane…

It kept re-adding because the flap of the aoa sensor was stuck in the "aircraft is pitching up" position.

Boeing management did not add a second sensor to some planes and also pilots were not aware of how to switch off mcas.

If you worked in large scale project you would know that most devs are assigned components to work on and usually have no idea what other components are or how they will interact.

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#133

Post by xiphias2 »

nobodyandproud wrote: post #85
meowface wrote: post #33

Earlier quoted context omitted.

For one, in part because many managers don't really understand software or software projects and think it's mostly just people * man-hours. "Through this cost-saving measure, we've tripled the amount of people working on the project and expect to reduce the amount of time it'll take to complete by two-thirds without incurring any additional costs." In their minds, it's an easy win. And they were probably regularly re…

Which means they have no business managing. It’s to tiresome to repeat because it should be obvious , but engineering is not a commodity. The end product may be, but not the design problem process nor the manufacturing process.

For me the solution is quite easy: I don't invest in companies where I don't see technical excellence happening.

The world is changing fast enough that old style management won't get investment after some time.

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#134

Post by nutjob2 »

ganeshkrishnan wrote: post #88
throwawayMUSE wrote: post #74

Earlier quoted context omitted.

I worked at a Fortune 10 in the early 2010s for several years. We had a small army of local and offshore Indian contractors through Larsen and Toubro Infotech, an Indian IT company. They were paid between $10-20 an hour, many couldn’t understand English. These guys have no CS background and would get stumped writing fizz buzz. But hey, management got ball game tickets and other favors from the L&TI reps. One of the d…

You must have been a pleasant employee to work with.

Actually, he sounds very good. The biggest problem in corporate environments is the amount of BS being spouted and people going along with said BS. When someone is actually honest and confronting issues, problems get fixed before they blow up. That's good for everyone.

Of course generally the reason people go along with the BS is that either they're incompetent or mediocre and BS suits them just fine, or they've given up and are just happy to be swept along and collect a paycheck.

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#135

Post by ganeshkrishnan »

nobodyandproud wrote: post #115
ganeshkrishnan wrote: post #87

Earlier quoted context omitted.

The software did exactly as it's supposed to do: If the AOA sensors tells you the plane is tilting too high up, pitch the nose down a little. The main issue was that there was only one AOA sensor connected in standard packages and the hardware for it failed. The pitot tubes failed for the doomed Air France incident which caused the software to read incorrectly and stall. For what it's worth AirBus has a huge engineer…

The problem is that developers also need to understand what they are coding, and fight back when they notice that the design is fundamentally flawed. In this case, the lack of redundancy should have been a red-flagged. This is why domain knowledge is at least as important as coding skills.

There were multiple redundancy built in. Multiple sensors, voting mechanism, manual override. Many of which meant re-training the pilots and Boeing wanted to avoid it.

I think southwest airlines had multiple sensors

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#136

Post by tarsinge »

crorella wrote: post #11

Money saved by firing experienced engs and outsourcing the development Why is that so hard to understand? Massive fail at risk management maybe?

Unfortunately it seems other critical industries are affected, reminded me of the EPR nuclear reactor fiasco https://en.wikipedia.org/wiki/EPR_(nuclear_reactor)#Olkiluot...

Olkiluoto (Finland)

> In May 2006, construction delays of about one year were announced, following quality control problems across the construction. In part the delays were due to the lack of oversight of subcontractors inexperienced in nuclear construction.

> At the end of June 2007, it was reported that Säteilyturvakeskus (STUK), the Finnish Radiation and Nuclear Safety Authority, had found a number of safety-related design and manufacturing 'deficiencies'.

> Initial cost estimates were about €3.7 billion, but the project has since seen several severe cost increments and delays, with latest cost estimates (from 2012) of more than €8 billion.

Flamanville (France)

> In April 2008, the French nuclear safety agency (Autorité de sûreté nucléaire, ASN) reported that a quarter of the welds inspected in the secondary containment steel liner are not in accordance with norms, and that cracks have been found in the concrete base.

> In April 2015, Areva informed the French nuclear regulator ASN that anomalies had been detected in the reactor vessel steel, causing "lower than expected mechanical toughness values". Further tests are underway.[64] In July 2015 The Daily Telegraph reported that Areva had been aware of this problem since 2006.[65] In June 2015, multiple faults in cooling system safety valves were discovered by ASN.

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#137

Post by gotdevops »

ganeshkrishnan wrote: post #132
dTal wrote: post #130

Earlier quoted context omitted.

This comment is glib and misinformed. Firstly, no the software did not do exactly as it's supposed to. It crashed the plane by rendering it unflyable. It's not supposed to do that. And your description of the MCAS specification is vastly over-simplified - it is not an autopilot and its job is to gradually change the behaviour of the plane in different flight modes (which it must detect), not to directly fly the plane…

It kept re-adding because the flap of the aoa sensor was stuck in the "aircraft is pitching up" position. Boeing management did not add a second sensor to some planes and also pilots were not aware of how to switch off mcas. If you worked in large scale project you would know that most devs are assigned components to work on and usually have no idea what other components are or how they will interact.

This is where testing comes in and from the fact packet it appears that the testing was also outsourced to another Indian firm?

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#138

Post by su30mki117 »

codingslave wrote: post #61

"In offices across from Seattle’s Boeing Field, recent college graduates employed by the Indian software developer HCL Technologies Ltd. occupied several rows of desks, said Mark Rabin, a former Boeing software engineer who worked in a flight-test group that supported the Max. The coders from HCL were typically designing to specifications set by Boeing. Still, “it was controversial because it was far less efficient t…

How are the points you have mentioned in your post relevant to the MCAS fault, which the article is supposed to be about?

It is explicitly mentioned in the article that the Indian companies DID NOT work on the faulty MCAS system - Boeing said the company did not rely on engineers from HCL and Cyient for the Maneuvering Characteristics Augmentation System, which has been linked to the Lion Air crash last October and the Ethiopian Airlines disaster in March. The Chicago-based planemaker also said it didn’t rely on either firm for another software issue disclosed after the crashes: a cockpit warning light that wasn’t working for most buyers.

This looks like a misleading article trying to shift blame away from Boeing management. Yes, there are problems with outsourcing, which the article brings up trying to show as reason for the problems with 737 Max.

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#139

Post by ranci »

la_barba wrote: post #41

I think this is just rabble-rousing. For all the flack outsourcing gets, if you hire an incompetent person, and they fuck up, you're still responsible for hiring an unqualified person. I don't think it matters if that person was outsourced or not. I haven't worked with any offshore teams, but friends tell me you can do what are called "client interviews" where the client, in this case Boeing, has the option of interv…

"I'm going to dismiss this because I don't like it"

Re: Boeing's 737 Max software outsourced to lower-paid engineers

#140

Post by ranci »

ganeshkrishnan wrote: post #90
la_barba wrote: post #41

I think this is just rabble-rousing. For all the flack outsourcing gets, if you hire an incompetent person, and they fuck up, you're still responsible for hiring an unqualified person. I don't think it matters if that person was outsourced or not. I haven't worked with any offshore teams, but friends tell me you can do what are called "client interviews" where the client, in this case Boeing, has the option of interv…

Absolutely rabble rousing. Given that it was the angle of attack hardware sensor which was stuck in the wrong position and giving out the wrong reading. The software got the reading that the nose was too pitched up and to prevent stall it pitched the nose down. The software could handle reading from multiple sensor which Boeing sold separately. It's an American management decision to make more money, not a $9 an hour…

Yeah, it was a hardware failure! God forbid the software engineers write defensive code in the case that hardware fails................
Post reply on HN