Live data from Hacker News

Microsoft President: We Need a Hippocratic Oath for Software Engineers

capitalandgrowth.org

341–350 of 544 posts

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#341
post #263

Earlier quoted context omitted.

So management bears no blame for requiring illegal work be done, on pain of termination? Said another way, engineers now need to be technical and legal experts in the business domain? (Remember employees in the US depend on the company for health insurance. Saying 'no' could cost a lot more than just ones position.) Most software engineers are not like doctors. We have little autonomy over what is created. Our respon…

Very few software engineers in the US, despite receiving health insurance as compensation, do not have sufficient cash comp to purchase health insurance on their own should they need to. I believe that this is a red herring. Yes, the drone operator that pushes the button is a murderer, even if it was the POTUS that gave the order for it to happen.

I don't think the drone operator is a good example. There are situations in which unethical things must happen, no matter what. If wars could be fought without killing we'd be doing that already.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#342
This has nothing to do with ethics and everything to do with hyping AI. “It’s so good you need to listen to me. Not just on technical matters but on moral matters. It’s too dangerous to give to just anyone but I can protect you from it. I can harness it for good. If only you would do as I say and give me $1b”

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#343
post #120

Earlier quoted context omitted.

"I was only following orders." Edit to add: If you are being asked to perform illegal or unethical acts as part of your employment, then perhaps termination is an ideal course of action? Unless of course your personal enrichment outweighs legalities or ethics in your worldview?

And I wasn't implying engineers should be entirely blameless. Everyone has a limited understanding of legal systems too complex for one person to fully grasp. And workers far below the level of decision makers should be judged according to evidence of their knowledge and responsibility. Likewise those who give orders should bear more responsibly. All these "companies take on a life of their own" arguments sound a lot…

> ...then we can make some sizes and structures illegal before they stumble/march into devastating incompetence.

Was with you until this part. Just hold them personally liable if someone gets hurt should they create an uncontrollable system and predictably fail to control it.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#344

The Hippocratic Oath works for doctors because they have power over what happens to their patient(s). Administrators and managers run the facility, but cannot tell Doctors what to do. It would only work for software engineers if we moved to the same model: Management run the facility but cannot tell software developers what to build. I don't think that's a viable model for managing software development.

I think that could work but the people working there wouldn't solely be software engineers (e.g. not just code monkeys). They would be a person using software engineering to solve a problem.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#345
post #206
post #181

Earlier quoted context omitted.

That's completely the wrong comparison and context. The reason MCAS came about was because management wanted to try to fudge a larger engine into an outdated design created for a different purpose rather than do the engineering and certification necessary for the new requirements and to update the system. Management wanted to save money. Of course the engineering leadership did not want to fudge something -- they wan…

Which demonstrates the irrelevancy of a hippocratic oath for engineers. Because the complexity of the system is such that no single engineer understands it all, and thus no single engineer is (or feels) responsible. Blaming it on management is also irrelevant, since management merely takes the advice of engineers, and do financial/business trade offs to maximize profit. If the engineers cannot tell that MCAS system c…

That's like saying burglars who break windows demonstrate the irrelevancy of door locks.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#346
post #80

Earlier quoted context omitted.

In sufficiently complex systems, you can always blame someone else if you're motivated to do so. https://www.youtube.com/watch?v=IssR_J0QWr4 But ultimately there's always a software engineer involved in the creation of software - and that's not true of any of the other roles you mentioned. Since software engineers are necessary and sufficient to produce software, they should always be held responsible, and any oath s…

How about a compromise then? Software engineers are responsible for the negative consequences of the software. Any and all responsibility they take is then also shared with every person that is above them in the hierarchy. Eg a developer does something and society finds this unethical and punishes them. The developer's boss, the boss's boss, the boss's boss's boss etc up to the CEO all get punished in the same way. F…

That makes no sense. Management just needs to decide on an ethics standard for the entire company and install an audit process that maintains that standard. That's the entire problem. Don't ask a handful of employees to do some charity.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#347
This is stupid. It's like asking a blacksmith to not build a specific knife, because it'll be used in a murder. You need to prosecute the murderer not the people building the knife.

Software, as an object, is harmless. It can be used for good or evil, just like anything else. So if anybody needs an oath is the people using the software, not the people building it.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#348
post #128

Earlier quoted context omitted.

Except that the start and goal are so far away you can legitimately claim you can't see the actual harm in writing something. You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. What if you start as something 100% ethical. But your company pivots to unethical application?

> You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. I mentioned this in another comment, but I'll say it again: Irrespective of any legal/ethical concerns, yes, I would like to know! If my boss just came to me and said "build a facial recognition system" I certainly will ask how it is going to be used. Not because I care about eth…

Why would your boss tell you about the unethical ways he is going to use software you have written?

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#349

Earlier quoted context omitted.

The programmer still decides whether that code gets written, since they’re the one writing it! If you write or review a piece of software, even if the spec was written by the PM/business, you’re endorsing whatever that spec says and all of its ethical implications. “Just following orders” is a famously poor defense at this point.

Exactly. I can’t believe all the blame-shifting I’m reading in this thread! It’s as if software engineers are suddenly these powerless victims, lacking agency over their work, only capable of saying “yes, boss, whatever you say, boss!” If a civil engineer’s manager told them to design an unsafe building or bridge, they’re not going to just say, “Sure thing manager! One death trap coming right up!” It is their ethical…

Imagine if your manager copied the safe bridge you designed with a magic replicator and now uses that exact same design somewhere else. You tell him that the bridge was not designed for this location and that the bridge will collapse in 5 years. Your boss fires you but you are still responsible for the collapse of the bridge.

Let's go further into absurdity. The engineer is kidnapping the daughter of the manager and blackmailing the manager to take the bridge down. Is it ethical to force someone else to be ethical even if its only possible through unethical means? What if there is a hero saves the daughter? Will the hero be liable for the collapsed bridge?

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#350

Earlier quoted context omitted.

Exactly. An oath means nothing: what you need is skin in the game. Some health practitioners are literally bought by Big Pharma, by their hospital accountant, etc. How would an oath fix that? Same with engineers or any other discipline. You need to make sure that everyone in the process has skin in the game. For me it's less about control (legislation) than about responsibility and accountability (assessments, eating…

What does “skin in the game” mean precisely in this context?

Having personal exposure, taking accountability for your actions. From Taleb of course.

The case of Snowden shows how bad it can turn when not everybody in the loop has skin in the game (asymmetry). Ironically, his behavior tells us he has been faithful to some kind of oath, but apparently none of his coworkers or supervisors.

Post reply on HN