Microsoft President: We Need a Hippocratic Oath for Software Engineers • Hacker News Board

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#161

Post by walrus01 » Mon, Aug 10, 2020, 5:03 AM UTC

pixl97 wrote: post #153

peterkelly wrote: post #102

Earlier quoted context omitted.

> Software engineers are accountable to their bosses before their users, no matter how high minded we like to pretend to be I've quit jobs in the past because of ethical concerns about the way in which those above me have been acting. In one case this involved bribery of senior government officials to push through a project that put at risk the privacy of hundreds of thousands of people. If you go along with shit lik…

And I mean, that is good of you to have a moral backbone, unfortunately there are many people behind you that will do the same job in software and they can be located anywhere in the world.

In the current state of things, that is factually true.

However the same argument could be applied to labor abuses in the textile/garment and shoe manufacturing industry. Most people who follow news are probably aware that about 20 years ago Nike, Adidas and other brands went through a period of terrible public relations disasters, after the working conditions in some of their shoe factories in developing nations were exposed by journalists.

The argument that could have been made at that time would also have been "well but if we don't employ these people, somebody else will just do the same thing with equally terrible labor/human rights violations somewhere else in the world, with even cheaper factories".

The situation today is not great, but it is significantly improved from how it was twenty years ago. There are third party neutral inspection/oversight agencies. Companies in the garment industry are forced to make public commitments to labor rights and reasonable working conditions, and to allow external auditing. They can't just hand wave away the problem and say "but if we don't do it , someone else will.."

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#162

Post by [deleted] » Mon, Aug 10, 2020, 5:06 AM UTC

tfehring wrote: post #141

Ygg2 wrote: post #128

Earlier quoted context omitted.

Except that the start and goal are so far away you can legitimately claim you can't see the actual harm in writing something. You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. What if you start as something 100% ethical. But your company pivots to unethical application?

This is a great point, and reasonable people can disagree about when an application's abuse or potential for abuse crosses the line. The same goes for pivots or for general-purpose code that's used elsewhere. (Is it ethical to contribute to internal tools at Facebook? What if those tools make other engineers way more effective at doing things that ultimately undermine democratic systems?) My point here isn't to dicta…

[deleted]

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#163

Post by cheez » Mon, Aug 10, 2020, 5:07 AM UTC

tfehring wrote: post #41

madrox wrote: post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

In sufficiently complex systems, you can always blame someone else if you're motivated to do so. https://www.youtube.com/watch?v=IssR_J0QWr4 But ultimately there's always a software engineer involved in the creation of software - and that's not true of any of the other roles you mentioned. Since software engineers are necessary and sufficient to produce software, they should always be held responsible, and any oath s…

I'd take the oath if there are no more dumbass interviews, I get paid like a surgeon and work 20 hours a week.

Until then, management falls on the sword, thanks.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#164

Post by spaced-out » Mon, Aug 10, 2020, 5:09 AM UTC

austincheney wrote: post #84

Someone1234 wrote: post #32

Doctors no longer take the Hippocratic Oath (because it is incompatible with a lot of difficult situations doctors are placed in). But more to the point: Why are we trying to shift focus from the wrongs large multi-nationals do to individual software engineers? Plus what would the result be if this "oath" conflicts with a manager's instructions? Maybe we should start with Microsoft, Google, Comcast, Oracle, and simil…

> Why are we trying to shift focus from the wrongs large multi-nationals do to individual software engineers? Because software engineers are the ones doing the actual work. By imposing an ethical standard on the people doing the real work the multinational executives then either accept that limitation or knowingly accept risks from intentionally violating the spirit of that limitation.

The people who actually decide to do these unethical things are the executives, not engineers. Executives are the ones who should be taking Hippocratic oaths.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#165

Post by smabie » Mon, Aug 10, 2020, 5:09 AM UTC

jojobas wrote: post #144

abj wrote: post #64

We need a Hippocratic Oath for those that manage and deploy capital. A Hippocratic Oath for only Software Engineers is addressing the symptom, not the cause. Most of us don't wake up and think, "How can I make user's life worse today?". We need to fix the incentives if we want to fix the behavior. We need to give Software Engineers writing spyware the option to say no and still feed their families. We need saying," T…

The key incentive is cash. As long as doing sinister things can pay, there will be people doing it.

Or maybe, just maybe, ethics isn't at all universal? If my time in other countries has taught me one thing, it's that different people and different cultures have very different ideas on right and wrong.

Ultimately (like, really ultimately), the very idea of ethics is a meaningless fabrication in the absence of God. Without an ultimate arbiter of truth, morality cannot exist.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#166

Post by MattGaiser » Mon, Aug 10, 2020, 5:09 AM UTC

The challenge here is the entire software engineering compensation and career model.

Let's consider Boeing. Aviation is complicated and takes a long time to learn. Good software engineers are constantly moving jobs to boost compensation, so Boeing would retain them for 1-2 years at most unless it were willing to radically increase its compensation year over year.

I have spent 1 year in my job at a parking software organization. I have learned very little about how the business works and couldn't tell you much about the software I work on beyond the features built while I have been there. Most of my team has been there just a year or two and they don't know either. Give me all the training you want, but I know very little about how any of this fits into the business model or how it is going to be used and I will probably never know before I leave. And if we use the median tenure of engineers for my organization, I am arguably scheduled to leave.

A friend is at a medium sized tech company. He spent 6 months on one project, spent 6 months on another, and now is interested in moving to a third. He works on "some cloud service, not quite sure which."

Another friend is just fed tickets, completes the tickets, and otherwise isn't even sure what his software is used for and what kind of companies buy it. He has been there a year.

So, sure, you can create an oath, but how many software engineers are going to ever know enough to object to something? It is maybe 50%?

To understand the bigger picture in many cases, engineers would need to stay on their projects to for 2-4 years. They would need to be there from the design phase to the implementation and deployment phase, as well as the day to day use phase. They would need to know a lot more about what the business actually is in many cases. In aviation, they would need to have engineers stay for 6-8 years.

How does that happen when 3 years at a company is considered a long time and plenty stay 1-2 years?

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#167

Post by tfehring » Mon, Aug 10, 2020, 5:10 AM UTC

paulryanrogers wrote: post #142

tfehring wrote: post #132

Earlier quoted context omitted.

> So management bears no blame for requiring illegal work be done, on pain of termination? No, I wouldn't say that. In many cases, management and engineering share the blame jointly and severally since they both have an opportunity to stop it. > Said another way, engineers now need to be technical and legal experts in the business domain? Engineers should know enough about their business domains to understand the eth…

> Engineers should know enough about their business domains to understand the ethical impacts of their work. This would be more reasonable in the era of 10-20 years in the same company or industry. Needing to job hop every 2-3 years for a decent raise, and software skills applying to a vast array of industries, makes it less reasonable IMO. > ...but I'm far less sympathetic for anyone making 5+ times the median US in…

> This would be more reasonable in the era of 10-20 years in the same company or industry. Needing to job hop every 2-3 years for a decent raise, and software skills applying to a vast array of industries, makes it less reasonable IMO.

Yeah, to be clear, I don't think software engineers have an infinite level of responsibility for understanding the ethical implications of their work. If you were a software engineer at a credit rating agency in 2006, and you didn't see the ethical dilemma because you didn't anticipate that contagion would be exacerbated by the shadow banking system to bring down the global economy, you get a pass. But if your prospective employer is, like, locking children in cages, or spreading disinformation on political candidates, you should probably find that out during the interview process.

> Not everyone here or in software makes that kind of money. Some of us in the Midwest--or who aren't as skilled at negotiating--don't pull down nearly that much.

Good point - I'm also in the Midwest and make less than that, for what it's worth. I've naturally had FAANG in mind as I type these comments, and more generally I think salaries for the more unethical roles tend to skew higher.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#168

Post by pixl97 » Mon, Aug 10, 2020, 5:13 AM UTC

austincheney wrote: post #66

It’s unfortunate, perhaps to the point of irrelevance, that a need for ethics was raised and then summarily limited to AI. Software, all software, needs an ethical standard and it currently has nothing remotely close in common practice. This will be a hard sell though, because most developers have no idea what ethics are, why they would be needed, or how they apply to practice. Frequently when the idea of ethics are…

Well, the only thing you 'need' at the end of the day to be a software engineer is a computer. It is probably the most accessible field that contains the term engineer in it. There are no physical artifacts necessary like most other engineering fields. More so only few software projects actually require any of those things you state above, such as medical devices or software that runs control systems that affect things in real life.

Does the same set if ethics/regulations even make sense for one person developing a guided missile versus another making a video game, that happens to shoot missiles at virtual targets?

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#169

Post by ryandrake » Mon, Aug 10, 2020, 5:14 AM UTC

pixl97 wrote: post #150

ryandrake wrote: post #127

Earlier quoted context omitted.

Exactly. I can’t believe all the blame-shifting I’m reading in this thread! It’s as if software engineers are suddenly these powerless victims, lacking agency over their work, only capable of saying “yes, boss, whatever you say, boss!” If a civil engineer’s manager told them to design an unsafe building or bridge, they’re not going to just say, “Sure thing manager! One death trap coming right up!” It is their ethical…

Eh, this isn't a great analogy either. If I'm an engineer that develops a single beam in a bridge, is it my fault if someone assembles those beams together in such a way that is dangerous? At which point does a function become unethical? Do you now have to only use software made in your country by ethical developers? Software isn't a bridge and comparisons fall apart quickly.

Wow, I’m realizing what an unpopular opinion this is on HN! Yes, as a software developer you should absolutely be accountable for the ethical concerns around what that protobuf you’re moving around from one API layer to the other is used for. You’re not a code monkey. Ask, and refuse if it’s unacceptable. I have quit jobs where the ultimate purpose of what I was building was evil.

EDIT:

Forum the original OP:

> Software engineers are accountable to their bosses before their users, no matter how high minded we like to pretend to be.

They are accountable to themselves and their own conscience before both their bosses and their users. I understand this is an uncomfortable line of thinking if your employer asks for ethically questionable project work, but I’d argue that if this is the case for you, it warrants career introspection.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#170

Post by tehsauce » Mon, Aug 10, 2020, 5:15 AM UTC

matz1 wrote: post #143

tehsauce wrote: post #38

"Engineering ethics" is a fallacy as it implies that engineers have high level influence on the goals set by organizations. If lockheed martin trained all of its missile engineers in ethics would they suddenly start building bombs that burst with flowers? Obviously not. Engineers are paid to design things against a very specific set of requirements. If an average engineer pushes back on ethical grounds they will simp…

Ethics are subjective and always evolving. I think Engineering ethics is not to tell you what to build or not to build but rather to tech you what is the likely outcome if you decided to build x or not to build x. Based on that its up to you to decide whatever to do or not do it.

I'm not sure what your engineering experience is, but I'm pretty certain that >80% of engineers are not in a position to refuse to work and quit their job without a severe impact on their career. And the same goes for managers and directors, all the way up to the top. Fail to explore a new technology your competitor is already working on? You've just damaged your company and all your employees. The only way to deal with these kinds of issues is regulation. Classic tragedy of the commons.