Live data from Hacker News

Microsoft President: We Need a Hippocratic Oath for Software Engineers

capitalandgrowth.org

471–480 of 544 posts

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#471
post #448
post #380

Earlier quoted context omitted.

>The MCAS is an optional component that reduces certification and training costs No. The MCAS was a "necessary" component for pitch stability - without it, a 737 MAX in a pitch-up attitude would, in the absence of correcting inputs, pitch up further and further until it stalled. Without it, the airframe is uncertifiable, full stop.

Cite? I'm certain that's not correct, everything I've read on it has said MCAS was specifically a software modifier put in place to allow the plane to respond substantially the same as a regular 737 without the larger engines, in order to avoid having to have additional training for all 737 pilots worldwide. Most aircraft, in a "pitch up attitude" will increase their angle of attack as thrust is applied. The issue wa…

Sure, a variety of references here: https://aviation.stackexchange.com/questions/73132/did-boein...

Although to be fair my summary wasn't entirely accurate - it wasn't that a MAX was outright dynamically unstable with no control input, as I described, but rather not sufficiently stable as to cause a monotonic increase in stick force as AOA increases, which can cause the combined system of pilot + flight dynamics to be unstable since the pilot relies on stick force as an indicator.

> Most aircraft, in a "pitch up attitude" will increase their angle of attack as thrust is applied

This is both incorrect and irrelevant. Most aircraft will climb when power is applied, but will not change their AOA unless the thrust axis is off-center. To a first approximation, power controls climb rate, and stick input changes AOA. Change in behavior under different power settings has little to do with the problem with the MAX. The problem with the MAX is that at high angles of attack - i.e., when the stick is held back, causing the air to meet the wing (and the engines) at a steeper angle - the engines, which are flung forward, start producing lift of their own and produce a pitch-up moment. This means that the further the pilot pulls the stick back, the less hard they have to pull. This is a dangerous inversion that increases the control order of the system, as it breaks the usual assumption that a given stick force will result in a given AOA, more or less.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#472
No, this is a meaningless sentiment. Floating a feel-good idea like this accomplishes little in the face of how the world actually works. In the real world the hippocratic oath does not prevent malpractice, over prescription of opioids, or name your beef with the entire healthcare system let alone doctors actions.

Engineers are hired to make weapons systems that kill civilians. Engineers make software that enable ubiquitous surveillance by whomever wants to use it for whatever purpose. No a Hippocratic oath will have a hard time influencing individuals and their organizations that’s what laws and the enforcement thereof are aimed at. I have little hope for oaths or laws however.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#473

Earlier quoted context omitted.

> These end your career. Agreed: https://news.ycombinator.com/item?id=24106617 > You don't need to be in any of them to write software. Then they are irrelevant. > Do we really want to impose that level of gatekeeping on the profession? Yes, absolutely. Everybody benefits except developers who probably shouldn't be writing software professionally in the first place.

The key to lawyers and doctors losing their licenses to practice is that they have professional bodies that will police/enforce the rules. However, they also have a monopoly to provide those services. I suspect corporations don't want limit software engineers to those with licenses only as there would be less of them available and costs of said licensed software engineers would increase.

> I suspect corporations don't want limit software engineers to those with licenses

It comes down to money, which includes liability. I suspect licensing won't ever happen in software unless law suits increase and licensing becomes a deterrent, at which point licensed developers will cost less after factoring for risk assessments. There are all kinds of unintended benefits for everybody that come from limiting negligence, which pays for itself. As a counter argument who benefits most from allowing the hiring and practicing of negligent developers?

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#474

Earlier quoted context omitted.

Engineers at the beam companies just certify that the beam meets its specs. I'm not sure why software would be any different. Bridges are complicated and made up of versatile submodules, just like software. Some other software engineer eventually designs the "bridge" and selects "beams" for the structure. If those beams fail to meet their specs, then the engineers who stood by them are at fault. If the bridge fails b…

What if the engineer isn't allowed to certify his beams to save on costs? That's basically what happened with 737 MAX.

Then the engineer designing the bridge should probably choose a different beam supplier who can stand by their product.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#475

So reading the comments. I am on the side of having business owners taking some kind of oath, not software engineers. If a software engineer was programming IE around the time that MS was hit by regulators years ago, and this "software engineer" oath was in place, would the software engineers being asked to code a web browser be at fault for the wrong-doings of the company? It just seems wrong for someone that has be…

There are not two sides, because both management and engineering should face consequences for building harmful software. And I'm not sure it's appropriate to test these "what ifs" as if looking for logical flaws in such a system, because a well-functioning court and jury should easily be able answer these questions, and I'm sure you had the "correct" answers in mind when you wrote them. Whether or not our courts and…

But what is harmful software? Today, while there are arguments about facebook being too addictive, it's really just competition between platforms, it's not everyone's opinion that the software engineers at facebook should be liable at this point for making an addictive algorithm. At some uncertain point in time, there's the possibility that facebook will be sued for the addictiveness and lose. Should that happen, I don't see why the software engineers should be partially to blame.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#476
post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

Doctors can have their licenses revoked. Software engineers cannot. There's no downside to being involved in unethical businesses.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#477
post #412

Earlier quoted context omitted.

> there wouldn't be any risk for any executive This isn't a real problem. I take it you have never worked outside of software. You don't need a law license to be the business manager of a major law firm or a medical license to manage a large hospital network. How often does such problems occur in major law firms or large hospital networks? In other words this is a hypothetical complaint of some future condition that…

> I take it you have never worked outside of software. You take it wrong, and reasoning by analogy with other fields is a very risky line of logic, because they're not at all similar in terms of internal regulation and professional autonomy. First of all, in fields like medicine, that kind of responsibility is necessary because clinical autonomy is far wider than any kind of professional autonomy that software engine…

Since absolutely every other professional industry employs some manner of uniform credentialing the only way opposing it in software, where is has not been tried in a meaningful way, makes sense is an argument from ignorance:

https://en.wikipedia.org/wiki/Argument_from_ignorance

Your bullet points conflate experimentation with production delivery. Licensed professionals in other industries can violate the ethical standards of practice so long as they are not billing for those violations and not representing a client with such violations.

> Second, one of the reasons why managing large hospital networks by people without a medical license works (more or less...) is that, for example, there's usually no legal way for the CEO to walk into an OR

So? I am a software developer and my bosses don't write software. No distinction.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#478

Earlier quoted context omitted.

> Software engineers are accountable to their bosses before their users, no matter how high minded we like to pretend to be. Isn't that what a hipppocratic oath would solve. They'd be accountable to the oath before their bosses, and that would give them reasonable grounds to refuse unethical work.

... and be an easy way to get rid of any engineer who fails to do the work asked of them. Think you can just get a new one?

The Hippocratic oath for software engineers should come along with legislation that makes it illegal to fire a software engineer for refusing to break the oath.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#479
post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

Being expected to behave ethically is part of being an Engineer regardless of what your boss expects. Now we increasingly see real world negative consequences of the work of software engineers I don't see why they should be any different.

In some Engineering societies maybe, in general you may be liable if you construct a building you shouldn't have.

Software is fundamentally different because until you run it it has no consequences, and even if you run it, it can be contained. I can write a worm and not release it on world. In that regard, it is more like engineering _plans_. I can draw up plans for a building that is designed to collapse with X number of persons inside -- in fact I can imagine either of the two assignments given as an exercise in University.

No reason to make more laws: it should be immaterial whether I chop down the Christmas tree at the local town square or program a robot to do it.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#480

Earlier quoted context omitted.

That's an unrelated problem. A person who follows an illegal order, whether in violation of law or ethics of practice, probably won't retain visa sponsorship long enough for that argument to matter. It also won't prevent them from suing their sponsor for wrongful termination and fraud. At any rate ethics is a hard sell to an industry whose professionals are completely unaware of what ethics are or how they apply.

I don't understand your argument. Do you think people who follow unethical behaviour are quickly fired? That is not usually the case if the unethical behaviour was profitable. Do you think that the legal system is quick to move so people won't be deported in 60 days? >At any rate ethics is a hard sell to an industry whose professionals are completely unaware of what ethics are or how they apply. This is simply not tr…

> Do you think people who follow unethical behaviour are quickly fired?

They are fired once the house of cards collapses in epic failure or when a scapegoat is needed. Those are both evidence of systemic failures.

Post reply on HN