Earlier quoted context omitted.
So management bears no blame for requiring illegal work be done, on pain of termination? Said another way, engineers now need to be technical and legal experts in the business domain? (Remember employees in the US depend on the company for health insurance. Saying 'no' could cost a lot more than just ones position.) Most software engineers are not like doctors. We have little autonomy over what is created. Our respon…
Very few software engineers in the US, despite receiving health insurance as compensation, do not have sufficient cash comp to purchase health insurance on their own should they need to. I believe that this is a red herring. Yes, the drone operator that pushes the button is a murderer, even if it was the POTUS that gave the order for it to happen.
Microsoft President: We Need a Hippocratic Oath for Software Engineers
341–350 of 544 posts
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#342Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#343Earlier quoted context omitted.
"I was only following orders." Edit to add: If you are being asked to perform illegal or unethical acts as part of your employment, then perhaps termination is an ideal course of action? Unless of course your personal enrichment outweighs legalities or ethics in your worldview?
And I wasn't implying engineers should be entirely blameless. Everyone has a limited understanding of legal systems too complex for one person to fully grasp. And workers far below the level of decision makers should be judged according to evidence of their knowledge and responsibility. Likewise those who give orders should bear more responsibly. All these "companies take on a life of their own" arguments sound a lot…
Was with you until this part. Just hold them personally liable if someone gets hurt should they create an uncontrollable system and predictably fail to control it.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#344The Hippocratic Oath works for doctors because they have power over what happens to their patient(s). Administrators and managers run the facility, but cannot tell Doctors what to do. It would only work for software engineers if we moved to the same model: Management run the facility but cannot tell software developers what to build. I don't think that's a viable model for managing software development.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#345Earlier quoted context omitted.
That's completely the wrong comparison and context. The reason MCAS came about was because management wanted to try to fudge a larger engine into an outdated design created for a different purpose rather than do the engineering and certification necessary for the new requirements and to update the system. Management wanted to save money. Of course the engineering leadership did not want to fudge something -- they wan…
Which demonstrates the irrelevancy of a hippocratic oath for engineers. Because the complexity of the system is such that no single engineer understands it all, and thus no single engineer is (or feels) responsible. Blaming it on management is also irrelevant, since management merely takes the advice of engineers, and do financial/business trade offs to maximize profit. If the engineers cannot tell that MCAS system c…
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#346Earlier quoted context omitted.
In sufficiently complex systems, you can always blame someone else if you're motivated to do so. https://www.youtube.com/watch?v=IssR_J0QWr4 But ultimately there's always a software engineer involved in the creation of software - and that's not true of any of the other roles you mentioned. Since software engineers are necessary and sufficient to produce software, they should always be held responsible, and any oath s…
How about a compromise then? Software engineers are responsible for the negative consequences of the software. Any and all responsibility they take is then also shared with every person that is above them in the hierarchy. Eg a developer does something and society finds this unethical and punishes them. The developer's boss, the boss's boss, the boss's boss's boss etc up to the CEO all get punished in the same way. F…
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#347Software, as an object, is harmless. It can be used for good or evil, just like anything else. So if anybody needs an oath is the people using the software, not the people building it.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#348Earlier quoted context omitted.
Except that the start and goal are so far away you can legitimately claim you can't see the actual harm in writing something. You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. What if you start as something 100% ethical. But your company pivots to unethical application?
> You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. I mentioned this in another comment, but I'll say it again: Irrespective of any legal/ethical concerns, yes, I would like to know! If my boss just came to me and said "build a facial recognition system" I certainly will ask how it is going to be used. Not because I care about eth…
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#349Earlier quoted context omitted.
The programmer still decides whether that code gets written, since they’re the one writing it! If you write or review a piece of software, even if the spec was written by the PM/business, you’re endorsing whatever that spec says and all of its ethical implications. “Just following orders” is a famously poor defense at this point.
Exactly. I can’t believe all the blame-shifting I’m reading in this thread! It’s as if software engineers are suddenly these powerless victims, lacking agency over their work, only capable of saying “yes, boss, whatever you say, boss!” If a civil engineer’s manager told them to design an unsafe building or bridge, they’re not going to just say, “Sure thing manager! One death trap coming right up!” It is their ethical…
Let's go further into absurdity. The engineer is kidnapping the daughter of the manager and blackmailing the manager to take the bridge down. Is it ethical to force someone else to be ethical even if its only possible through unethical means? What if there is a hero saves the daughter? Will the hero be liable for the collapsed bridge?
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#350Earlier quoted context omitted.
Exactly. An oath means nothing: what you need is skin in the game. Some health practitioners are literally bought by Big Pharma, by their hospital accountant, etc. How would an oath fix that? Same with engineers or any other discipline. You need to make sure that everyone in the process has skin in the game. For me it's less about control (legislation) than about responsibility and accountability (assessments, eating…
What does “skin in the game” mean precisely in this context?
The case of Snowden shows how bad it can turn when not everybody in the loop has skin in the game (asymmetry). Ironically, his behavior tells us he has been faithful to some kind of oath, but apparently none of his coworkers or supervisors.