Microsoft President: We Need a Hippocratic Oath for Software Engineers • Hacker News Board

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#141

Post by tfehring » Mon, Aug 10, 2020, 4:37 AM UTC

Ygg2 wrote: post #128

tfehring wrote: post #111

Earlier quoted context omitted.

The programmer still decides whether that code gets written, since they’re the one writing it! If you write or review a piece of software, even if the spec was written by the PM/business, you’re endorsing whatever that spec says and all of its ethical implications. “Just following orders” is a famously poor defense at this point.

Except that the start and goal are so far away you can legitimately claim you can't see the actual harm in writing something. You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. What if you start as something 100% ethical. But your company pivots to unethical application?

This is a great point, and reasonable people can disagree about when an application's abuse or potential for abuse crosses the line. The same goes for pivots or for general-purpose code that's used elsewhere. (Is it ethical to contribute to internal tools at Facebook? What if those tools make other engineers way more effective at doing things that ultimately undermine democratic systems?)

My point here isn't to dictate what software is or isn't ethical, but to argue that if a program is unethical, its ethical implications are the responsibility of the engineer(s) who wrote it.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#142

Post by paulryanrogers » Mon, Aug 10, 2020, 4:38 AM UTC

tfehring wrote: post #132

paulryanrogers wrote: post #103

Earlier quoted context omitted.

So management bears no blame for requiring illegal work be done, on pain of termination? Said another way, engineers now need to be technical and legal experts in the business domain? (Remember employees in the US depend on the company for health insurance. Saying 'no' could cost a lot more than just ones position.) Most software engineers are not like doctors. We have little autonomy over what is created. Our respon…

> So management bears no blame for requiring illegal work be done, on pain of termination? No, I wouldn't say that. In many cases, management and engineering share the blame jointly and severally since they both have an opportunity to stop it. > Said another way, engineers now need to be technical and legal experts in the business domain? Engineers should know enough about their business domains to understand the eth…

> Engineers should know enough about their business domains to understand the ethical impacts of their work.

This would be more reasonable in the era of 10-20 years in the same company or industry. Needing to job hop every 2-3 years for a decent raise, and software skills applying to a vast array of industries, makes it less reasonable IMO.

> ...but I'm far less sympathetic for anyone making 5+ times the median US income.

Not everyone here or in software makes that kind of money. Some of us in the Midwest--or who aren't as skilled at negotiating--don't pull down nearly that much.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#143

Post by matz1 » Mon, Aug 10, 2020, 4:41 AM UTC

tehsauce wrote: post #38

"Engineering ethics" is a fallacy as it implies that engineers have high level influence on the goals set by organizations. If lockheed martin trained all of its missile engineers in ethics would they suddenly start building bombs that burst with flowers? Obviously not. Engineers are paid to design things against a very specific set of requirements. If an average engineer pushes back on ethical grounds they will simp…

Ethics are subjective and always evolving. I think Engineering ethics is not to tell you what to build or not to build but rather to tech you what is the likely outcome if you decided to build x or not to build x. Based on that its up to you to decide whatever to do or not do it.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#144

Post by jojobas » Mon, Aug 10, 2020, 4:42 AM UTC

abj wrote: post #64

We need a Hippocratic Oath for those that manage and deploy capital. A Hippocratic Oath for only Software Engineers is addressing the symptom, not the cause. Most of us don't wake up and think, "How can I make user's life worse today?". We need to fix the incentives if we want to fix the behavior. We need to give Software Engineers writing spyware the option to say no and still feed their families. We need saying," T…

The key incentive is cash. As long as doing sinister things can pay, there will be people doing it.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#145

Post by leeoniya » Mon, Aug 10, 2020, 4:43 AM UTC

ezoe wrote: post #113

tfehring wrote: post #41

Earlier quoted context omitted.

In sufficiently complex systems, you can always blame someone else if you're motivated to do so. https://www.youtube.com/watch?v=IssR_J0QWr4 But ultimately there's always a software engineer involved in the creation of software - and that's not true of any of the other roles you mentioned. Since software engineers are necessary and sufficient to produce software, they should always be held responsible, and any oath s…

The problem is, software can be really complex it's possible to make it so no one programmer can understand the whole picture. The tasks can be divided so the individual programmers are given order like "do X in condition of Y" which itself seems harmless and lawful, but combining them lead to malicious behaviours.

this is precisely the problem. few physicists knew they were all collectively building a nuclear bomb because the big picture was well compartmentalized. feynman only found out because he picked the locks of his colleagues to piece it all together.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#146

Post by SpicyLemonZest » Mon, Aug 10, 2020, 4:44 AM UTC

optimiz3 wrote: post #81

asperous wrote: post #31

Earlier quoted context omitted.

I agree, however not all ethics can be legislated. This sort of thing works in other professions like medicine because malpractice can cause doctors to lose their license. Same with civil engineers. This changes things because the choice is now quitting or possibly never being able to work in the field again. Perhaps principal software engineers in charge of life or death software should be licensed for accountabilit…

Doctors don't typically have bosses directing how they treat patients; Doctors have full agency, front line grunt workers do not.

But that's not an independent fact. That's one of the major goals of medical oaths; they establish that doctors must be able to determine the right course of action on their own, that ethical doctors won't work for an organization that won't give them that agency.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#147

Post by hyko » Mon, Aug 10, 2020, 4:45 AM UTC

It should be noted that no ethically-trained software engineer would ever consent to write a DestroyBaghdad procedure. Basic professional ethics would instead require him to write a DestroyCity procedure, to which Baghdad could be given as a parameter. -Nathaniel Borenstein

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#148

Post by wrnr » Mon, Aug 10, 2020, 4:45 AM UTC

Why are people so obsessed with "don't be evil" if "be good" is a much more effective commandment. This suit had to visit pope Francis to ask him how to program a soulless machine, his time would have been better spend helping a local charity or even better a struggling family member.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#149

Post by logicchains » Mon, Aug 10, 2020, 4:45 AM UTC

tfehring wrote: post #132

paulryanrogers wrote: post #103

Earlier quoted context omitted.

So management bears no blame for requiring illegal work be done, on pain of termination? Said another way, engineers now need to be technical and legal experts in the business domain? (Remember employees in the US depend on the company for health insurance. Saying 'no' could cost a lot more than just ones position.) Most software engineers are not like doctors. We have little autonomy over what is created. Our respon…

> So management bears no blame for requiring illegal work be done, on pain of termination? No, I wouldn't say that. In many cases, management and engineering share the blame jointly and severally since they both have an opportunity to stop it. > Said another way, engineers now need to be technical and legal experts in the business domain? Engineers should know enough about their business domains to understand the eth…

> Ethics and law are orthogonal, so thankfully this is generally much easier than being a legal expert.

I suspect a lot of philosophers would disagree with you that ethics is much simpler than interpreting laws.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#150

Post by pixl97 » Mon, Aug 10, 2020, 4:46 AM UTC

ryandrake wrote: post #127

tfehring wrote: post #111

Earlier quoted context omitted.

The programmer still decides whether that code gets written, since they’re the one writing it! If you write or review a piece of software, even if the spec was written by the PM/business, you’re endorsing whatever that spec says and all of its ethical implications. “Just following orders” is a famously poor defense at this point.

Exactly. I can’t believe all the blame-shifting I’m reading in this thread! It’s as if software engineers are suddenly these powerless victims, lacking agency over their work, only capable of saying “yes, boss, whatever you say, boss!” If a civil engineer’s manager told them to design an unsafe building or bridge, they’re not going to just say, “Sure thing manager! One death trap coming right up!” It is their ethical…

Eh, this isn't a great analogy either. If I'm an engineer that develops a single beam in a bridge, is it my fault if someone assembles those beams together in such a way that is dangerous? At which point does a function become unethical? Do you now have to only use software made in your country by ethical developers?

Software isn't a bridge and comparisons fall apart quickly.