Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft President: We Need a Hippocratic Oath for Software Engineers

capitalandgrowth.org

Post reply on HN
Print view View original

481–490 of 544 posts

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#481

Post by eat_veggies »

coding123 wrote: post #475
eat_veggies wrote: post #460

Earlier quoted context omitted.

There are not two sides, because both management and engineering should face consequences for building harmful software. And I'm not sure it's appropriate to test these "what ifs" as if looking for logical flaws in such a system, because a well-functioning court and jury should easily be able answer these questions, and I'm sure you had the "correct" answers in mind when you wrote them. Whether or not our courts and…

But what is harmful software? Today, while there are arguments about facebook being too addictive, it's really just competition between platforms, it's not everyone's opinion that the software engineers at facebook should be liable at this point for making an addictive algorithm. At some uncertain point in time, there's the possibility that facebook will be sued for the addictiveness and lose. Should that happen, I d…

If Facebook gets sued for its addictiveness and loses, then that would be a pretty solid judgment of its harm, wouldn't it? And it'd be up to the court to decide whether the engineers (and which ones) were complicit. There is room for nuance here.

> I don't see why the software engineers should be partially to blame.

Because they chose to build it! Software engineers aren't ignorant and helpless executors of instructions. They are responsible for what they build.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#482

Post by ryandrake »

jon-wood wrote: post #279
bcrosby95 wrote: post #248

Earlier quoted context omitted.

So the people that wrote java.util.List are liable for all unethical software that makes use of java.util.List?

No, of course they're not. Just like the scientists that developed morphine aren't responsible for a doctor later using it to kill a patient.

Sure, same as the LLVM example someone else pointed out. Good points. I’ll qualify my opinion then. To the extent that the engineer can know the ultimate application of their work, he or she should be responsible for ensuring it is being used ethically.

So, the engineer writing a binary search, knowingly working on “Project Orbital Death Ray” or “Voter Suppression 2.1” should know better. I hope we can at least agree on that one.

The engineer writing a linked list or moving around Protobufs for their some open source toolset gets a pass because their project as they understand it is ethically neutral. BUT there will be that engineer who then takes those tools and integrates it into “Project Orbital Death Ray”. That’s maybe where accountability should begin.

Everyone’s talking about the managers taking the blame and yes they’re culpable too. But at the end of the day an actual software developer’s fingers type the code in. If that developer knows what he is working on, he needs to bear the responsibility, too.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#483

Post by pc86 »

imtringued wrote: post #329
skissane wrote: post #118

Earlier quoted context omitted.

> Said another way, engineers now need to be technical and legal experts in the business domain? Consider something like the 737 MAX debacle – did the programmers writing the MCAS code actually have enough aviation domain knowledge and understanding of where the component fit in the overall system to realise it was a threat to people's lives? I don't know, but my guess is the most likely answer is "No".

From my limited information the MCAS code was primarily causing problems in association of incorrect readings by damaged sensors. Of course one could argue that this is an engineering failure because the MCAS failed to account for wrong sensor input but when you consider the legal implications of a MCAS fallback there is actually not much that can be done on the software side. The MCAS is an optional component that r…

> If the software engineer was under a hippocratic oath then he would have to refuse to build the MCAS entirely but not because the idea of an MCAS is inherently unethical, no, he would have to refuse because the company he works at wants to use the MCAS for a non ethical purpose (namely operate and hide the existence of MCAS even when it is unsafe to do so).

One, it's not going to be clear from the request that the MCAS would want to be used in unethical ways.

Since the Hippocratic oath is the argument here, how many software developers want to work in a system closer to physicians? A national cartel controlling membership and licensure - tough luck if you want to hire more developers because there's an artificially limited supply. Mandatory academic training - goodbye self-taught developers. Follow-on training with pay 1/5th or less of your attending physicians - I know residents in specialties where attendings are paid $500k a year to start, and they are making $60k a year. Brutal shift work - residents work 70-80 hours a week easily. Toxic leadership - I've heard horror stories of residents being forced to lie on ACGME forms regarding their hours under penalty of being outright fired from their residency slot, which would make it nearly impossible to get a job as a physician (mainly because you'd have to apply to a different residency program and explain your termination).

I know they're not suggesting bringing the entire medical education & training structure over to tech workers, and everyone here likes to think that they're brilliant and changing lives every day but most of us are just throwing shitcode JS into a computer for 3-4 hours a day for an ad tech company and not much more. The comparison falls apart pretty quickly.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#484

Post by GordonS »

mcv wrote: post #409
madrox wrote: post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

The oath should be for everybody involved in the process. After the 2008 mortgage crisis, Netherland required everybody working at banks to take the banker's oath, which is mostly about balancing the interests of the 4 main stakeholders of the bank: shareholders, customers, employee, and society. It's pretty broad, it doesn't magically fix everything, but it does make everybody more aware of their responsibilities. M…

Where do you draw the line though? Does everyone employed in any capacity at any kind of business have to take an oath?

It seems to me that if everyone has to promise to do no evil, the meaning of such oaths would become diluted.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#485

Post by SkyPuncher »

Someone1234 wrote: post #32

Doctors no longer take the Hippocratic Oath (because it is incompatible with a lot of difficult situations doctors are placed in). But more to the point: Why are we trying to shift focus from the wrongs large multi-nationals do to individual software engineers? Plus what would the result be if this "oath" conflicts with a manager's instructions? Maybe we should start with Microsoft, Google, Comcast, Oracle, and simil…

> Doctors no longer take the Hippocratic Oath

Lol. I'm married to an MD. Doctors definitely still take the Hippocratic Oath. Perhaps some places no longer do it, but I am not aware of any of here peers that have not taken the Hippocratic Oath.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#487

Post by doorstar »

eat_veggies wrote: post #460
coding123 wrote: post #443

So reading the comments. I am on the side of having business owners taking some kind of oath, not software engineers. If a software engineer was programming IE around the time that MS was hit by regulators years ago, and this "software engineer" oath was in place, would the software engineers being asked to code a web browser be at fault for the wrong-doings of the company? It just seems wrong for someone that has be…

There are not two sides, because both management and engineering should face consequences for building harmful software. And I'm not sure it's appropriate to test these "what ifs" as if looking for logical flaws in such a system, because a well-functioning court and jury should easily be able answer these questions, and I'm sure you had the "correct" answers in mind when you wrote them. Whether or not our courts and…

> a well-functioning court and jury should easily be able answer these questions,

I'm going to ignore the qualifier of 'well functioning' which obviously up for debate. The process of being charged with a crime, being put on trial, wondering at the consequences of the outcome etc., is no joke. It is a tremendously time-consuming, expensive, and stressful processes and even if you are acquitted there is no undoing the damage that has been done. There's a reason doctor's spend huge chunks of their income on malpractice insurance, and if we decide that engineers need the same protection in case they get sued than the biggest beneficiary is going to be the insurance companies.

If insurance companies also had to sigh oaths we might make some progress, but the nature of their game is to spread the risk - which is to say they take money from a lot of people and hope they never have to pay them back. There's only so much regulation can do about that.

It shouldn't just be thrown out as 'well if you make a decision in good faith then you are sure to win your court case'. It is not a reasonable burden to put on someone who cannot anticipate all the possible outcomes of decisions they make.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#489

Post by coliveira »

This is another example of the system trying to move to individual workers the failures of the economic elite. It is not the software engineering profession that is putting in risk the world, instead it is the unscrupulous drive for more profits coming from investors and CEOs.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#490

Post by xdavidliu »

walrus01 wrote: post #40
madrox wrote: post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

> Software engineers are accountable to their bosses before their users, no matter how high minded we like to pretend to be. I agree to a very limited extent about the hierarchical nature of a typical corporation, but I also disagree. Software engineers at a certain level of their career and with relatively uncommon skills can pick and choose what companies they want to work at. In my opinion people of good moral cha…

> Software engineers at a certain level of their career and with relatively uncommon skills can pick and choose what companies they want to work at.

What percent of software engineers does this statement apply to?

Post reply on HN