Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft President: We Need a Hippocratic Oath for Software Engineers

capitalandgrowth.org

Post reply on HN
Print view View original

321–330 of 544 posts

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#321

Post by watt »

threatripper wrote: post #78

I say no. My boss says "well, then no more salary for you". I say yes. My boss smiles. I feel dirty but my kids are not hungry.

Think more like a member of guild.

In fact, I am starting to think certifications like PCI DSS are similar to oath: "I will not leak customer data, I will not leak card information". If the oath is broken, the PCI retaliation is swift: the whole company is totally cut off from processing payments. The company compliance is ensured on threat of bankruptcy.

So you push back: pushing this through means the company will fail PCI DSS audit once that comes up, and then we're all out of jobs. No salary for anyone. And the boss thinks again.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#323

Post by Leherenn »

mar77i wrote: post #278
puzzledobserver wrote: post #155

Is there evidence that doctors and lawyers are, by virtue of their oaths, codes of conduct, and background study of ethics, more ethical, moral, and upright than members of the general population? Also, whose ethics would such a Hippocratic Oath advance? For every privacy-conscious person saying that encryption-everywhere is good, there is a law enforcement officer speaking of reduced abilities to solve crimes. Imagi…

> Imagine a software engineer who has been asked to place a backdoor in some software. I imagine such an engineer just gets a development plan and doesn't get to see the bigger picture, implying the backdoor. It might only get enabled on integration into a larger codebase, and nobody out of the loop will be able to extrapolate its existence from what they get to know for sure. Hence I completely agree with the argume…

I disagree. I only have one point of data from past experience (not a backdoor, but working on a potentially unethical system), but I would say most developers know exactly what they are doing, or they know deep down but don't try to clarify in order to absolve themselves.

If we go back to backdoors, yes, sometimes, in the simplest cases (e.g. a "root" account), it might get in prod through trickery. But anything more complex and you need to know what you are trying to achieve.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#325

Post by wizzwizz4 »

logicchains wrote: post #149
tfehring wrote: post #132

Earlier quoted context omitted.

> So management bears no blame for requiring illegal work be done, on pain of termination? No, I wouldn't say that. In many cases, management and engineering share the blame jointly and severally since they both have an opportunity to stop it. > Said another way, engineers now need to be technical and legal experts in the business domain? Engineers should know enough about their business domains to understand the eth…

> Ethics and law are orthogonal, so thankfully this is generally much easier than being a legal expert. I suspect a lot of philosophers would disagree with you that ethics is much simpler than interpreting laws.

You can derive most ethics in a couple of decades, plus everyday experience. Try deriving the law from that.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#326

Post by Chris2048 »

watt wrote: post #314
MattGaiser wrote: post #133

Earlier quoted context omitted.

I can’t imagine software engineers staying in a company long enough to acquire it. For this to work, software engineer compensation would need to change radically.

Maybe that's the point of this oath? Software engineers will refuse to work on these areas, unless demands of the oath can be met.

If a doctor says no, it's because of legal liability and risk to licence.

For the same reason, it's harder to hire some rando budget doctor because the field is gate-keeped by the requirement of a licence, and liability.

You can't magically make Engineering the same without the same conditions. Add barriers top entry that see my pay rise, or have cheap programmers with no liability.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#327

Post by throwaways885 »

creato wrote: post #106

I think a lot of the recent focus on "AI ethics" is just self-aggrandizement. I also think a lot of people here are overestimating how broad the agreement is on what is ethical and what is not (and thus what would be prevented from happening under such a "Hippocratic Oath"). Examples being: Microsoft/GitHub's work with ICE, Google's China search engine, use of AI in military applications I personally have a strong op…

For each of those examples, there are also positive social justice arguments if you look at them in different directions:

- Microsoft/GitHub's work with ICE [prevents illegal migration, saving those workers from a life of wage slavery]

- Google's China search engine [gives more information than otherwise would've been available]

- use of AI in military applications [can save lives by no longer bombing civilians where it can be avoided]

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#328

Post by austincheney »

alxlaz wrote: post #244
austincheney wrote: post #84

Earlier quoted context omitted.

> Why are we trying to shift focus from the wrongs large multi-nationals do to individual software engineers? Because software engineers are the ones doing the actual work. By imposing an ethical standard on the people doing the real work the multinational executives then either accept that limitation or knowingly accept risks from intentionally violating the spirit of that limitation.

> accept risks from intentionally violating the spirit of that limitation. What risk? Since it's the engineers, not the executives that take any oath of professional conduct, there wouldn't be any risk for any executive. All this would get us is a legal framework for throwing engineers under the bus via an ethical commission if they do something silly like blowing the whistle on an unethical decision higher up in the…

> there wouldn't be any risk for any executive

This isn't a real problem. I take it you have never worked outside of software. You don't need a law license to be the business manager of a major law firm or a medical license to manage a large hospital network. How often does such problems occur in major law firms or large hospital networks?

In other words this is a hypothetical complaint of some future condition that doesn't exist in order to avoid accepting responsibility in the present.

EDIT:

Also see: https://en.wikipedia.org/wiki/Sarbanes%E2%80%93Oxley_Act

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#329

Post by imtringued »

skissane wrote: post #118
paulryanrogers wrote: post #103

Earlier quoted context omitted.

So management bears no blame for requiring illegal work be done, on pain of termination? Said another way, engineers now need to be technical and legal experts in the business domain? (Remember employees in the US depend on the company for health insurance. Saying 'no' could cost a lot more than just ones position.) Most software engineers are not like doctors. We have little autonomy over what is created. Our respon…

> Said another way, engineers now need to be technical and legal experts in the business domain? Consider something like the 737 MAX debacle – did the programmers writing the MCAS code actually have enough aviation domain knowledge and understanding of where the component fit in the overall system to realise it was a threat to people's lives? I don't know, but my guess is the most likely answer is "No".

From my limited information the MCAS code was primarily causing problems in association of incorrect readings by damaged sensors. Of course one could argue that this is an engineering failure because the MCAS failed to account for wrong sensor input but when you consider the legal implications of a MCAS fallback there is actually not much that can be done on the software side.

The MCAS is an optional component that reduces certification and training costs. It is definitively possible to fly the plane without accidents even with a disabled MCAS. So why can't the MCAS be turned off automatically when sensors fail? Because that changes the classification of the plane and therefore requires pilots to be certified for a new machine and receive new flight training for both MCAS and no MCAS modes.

If the software engineer was under a hippocratic oath then he would have to refuse to build the MCAS entirely but not because the idea of an MCAS is inherently unethical, no, he would have to refuse because the company he works at wants to use the MCAS for a non ethical purpose (namely operate and hide the existence of MCAS even when it is unsafe to do so).

This is basically a reverse audit but the software engineer has no authority conduct such an audit and even if he was allowed to, the business has no obligation to give him the necessary information to determine whether the MCAS will be used unethically.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#330

Post by austincheney »

fnord123 wrote: post #255
austincheney wrote: post #84

Earlier quoted context omitted.

> Why are we trying to shift focus from the wrongs large multi-nationals do to individual software engineers? Because software engineers are the ones doing the actual work. By imposing an ethical standard on the people doing the real work the multinational executives then either accept that limitation or knowingly accept risks from intentionally violating the spirit of that limitation.

This is not realistic. Any H1NB visa holder who doesn't do what they are told will be fired and must move their family back to their country of origin. SWE would need to first unionize to protect workers from being fired/deported for pushing back before anything like this is even considered. Or tackle the problem where it begins: with the organization.

That's an unrelated problem. A person who follows an illegal order, whether in violation of law or ethics of practice, probably won't retain visa sponsorship long enough for that argument to matter. It also won't prevent them from suing their sponsor for wrongful termination and fraud.

At any rate ethics is a hard sell to an industry whose professionals are completely unaware of what ethics are or how they apply.

Post reply on HN