Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microsoft President: We Need a Hippocratic Oath for Software Engineers

capitalandgrowth.org

Post reply on HN
Print view View original

491–500 of 544 posts

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#491

Post by coliveira »

tfehring wrote: post #41
madrox wrote: post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

In sufficiently complex systems, you can always blame someone else if you're motivated to do so. https://www.youtube.com/watch?v=IssR_J0QWr4 But ultimately there's always a software engineer involved in the creation of software - and that's not true of any of the other roles you mentioned. Since software engineers are necessary and sufficient to produce software, they should always be held responsible, and any oath s…

The big question is: who has ultimate visibility on the consequences of a particular project? Very frequently software engineers are asked to work on projects where they only know one side of the picture. The executives in the company are the ones who know the ultimate context of what they're doing.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#492

Post by blunte »

Mandatum wrote: post #290
blunte wrote: post #288

Why should this concept be limited to just AI? How many developers knowingly enabled or directly wrote software for predatory companies like Facebook, (sometimes/mostly?) Google, sometimes Apple, historically often Microsoft, many of the large banks, Verizon and ATT and Comcast (and probably other major telcos), debt collection agencies, payday lenders, and on and on. I'm quite certain that many HN readers knowingly…

So true. I'd say much, much more harm has been done by regular CRUD programmers than any others. You really think AI is going to introduce regulation around this? They're just going to treat it like munitions, because that's all the lawmakers know.

I suspect this is somewhat a problem of age and experience. When you are younger (at least typically speaking... ignoring the rare exceptional people), you get caught up in the job at hand. It is new to you, and the environment is exciting. You focus your attention and your effort narrowly.

If you have someone to shake you from your myopia, or you get older and start to gain perspective, you question your efforts and their potential outcomes.

To be fair though, at the lowest CRUD level, it's no worse than someone who builds roads or cooks food. We cannot reasonably consider nor police the use of those roads or the meals we make.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#493

Post by kelvin0 »

tfehring wrote: post #41
madrox wrote: post #8

I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…

In sufficiently complex systems, you can always blame someone else if you're motivated to do so. https://www.youtube.com/watch?v=IssR_J0QWr4 But ultimately there's always a software engineer involved in the creation of software - and that's not true of any of the other roles you mentioned. Since software engineers are necessary and sufficient to produce software, they should always be held responsible, and any oath s…

Here's how this oath thing would play out:

1) New regulation forces some branches of software engineering to have some type of oath.

2) Now some software jobs will require only oath takers to do.

3) A well payed and powerful new cast of software engineers is born.

4) They are highly paid and have a powerful lobby working for them.

5) The oath takers become very frisky and only work on jobs with minimal risk. The ones that do screw up have an armada of lawyers, because of course they have a new association with deep pockets.

6) Innovation stalls for a while.

7) Big corps start outsourcing some of the oath-taking jobs. These engineers are not bound by the same regulation. Screw ups happen, people die at some point.

8) Maybe we should have the outsourced engineers also take an oath? Back to square 1

This is exactly what I found happened for medical Doctors in Canada (don't know for US). Not saying doctors are not doing a good job, and I can't imagine the stress and pressure they operate from. But suing for malpractice in Canada can be challenging to say the least. I have personal account of a Family member who was grossly mistreated, and all the Doctor did was changed hospitals, nothing more than a slap on the wrist.

https://diamondlaw.ca/blog/how-canadian-law-discourages-pati...

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#494

Post by BeetleB »

Ygg2 wrote: post #299
BeetleB wrote: post #175

Earlier quoted context omitted.

> You write a tool for let's say recognizing faces. Will it be used for login onto computer? Tracking dissidents? IDing corpses? Who knows. I mentioned this in another comment, but I'll say it again: Irrespective of any legal/ethical concerns, yes, I would like to know! If my boss just came to me and said "build a facial recognition system" I certainly will ask how it is going to be used. Not because I care about eth…

> If they pivot after my work is done, I won't feel responsible. So if you invented dynamite you wouldn't feel responsible for its use? But, let's change it a bit more personal. You write an awesome OSS yaml parser. It's so good, that GFW of China uses it as a main component, and this gets published in the news. What would you do? Nothing you did changed, but suddenly your work is powering an unethical component.

> So if you invented dynamite you wouldn't feel responsible for its use?

I would if I were inventing dynamite, but that's not what this scenario is.

A person working for a knife manufacturer need not worry about it being used for murder, as that's not what the primary use. And facial recognition is a lot less harmful than even that.

Trust me: I work for a company that produces certain goods used for all kinds of good and nefarious purposes depending on who buys it. My conscience is clear.

> But, let's change it a bit more personal. You write an awesome OSS yaml parser. It's so good, that GFW of China uses it as a main component, and this gets published in the news.

> What would you do? Nothing you did changed, but suddenly your work is powering an unethical component.

I wouldn't do anything:

1. This is milder than the knife scenario above. Of course I don't care if people use it in a poor way - unless there is a straightforward technical mitigation I could do. In your example, given that the source code is available, that is not an option.

2. There's a certain hypocrisy in releasing something as open source and then complaining about how it is used. If it bothers you, then modify your license!

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#495

Post by jki275 »

dTal wrote: post #471
jki275 wrote: post #448

Earlier quoted context omitted.

Cite? I'm certain that's not correct, everything I've read on it has said MCAS was specifically a software modifier put in place to allow the plane to respond substantially the same as a regular 737 without the larger engines, in order to avoid having to have additional training for all 737 pilots worldwide. Most aircraft, in a "pitch up attitude" will increase their angle of attack as thrust is applied. The issue wa…

Sure, a variety of references here: https://aviation.stackexchange.com/questions/73132/did-boein... Although to be fair my summary wasn't entirely accurate - it wasn't that a MAX was outright dynamically unstable with no control input, as I described, but rather not sufficiently stable as to cause a monotonic increase in stick force as AOA increases, which can cause the combined system of pilot + flight dynamics to b…

Right -- it didn't give the exact same feedback to the pilot that the regular 737 did, which was why MCAS was created. The aircraft is no more or less unstable than a regular 737.

The original 737 does exactly the same thing the Max does with respect to producing a pitch-up moment -- as does nearly every other aircraft. It's just not nearly as pronounced as the Max is.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#496

Post by drivingmenuts »

kazagistar wrote: post #430
drivingmenuts wrote: post #243

Earlier quoted context omitted.

Because it allows people to associate with whom they choose to. Remove that and you’ve opened the gate to legal racism, legally institutionalized homophobia, banning of religion; the list is endless. The five freedoms are the pillars of our Constitution. Without them, we are no better than China or Russia or even any third-world hellhole you care to mention.

I don't see that even a little bit, your cause effect isn't explained. I should phrase it differently. Why is an absolute freedom of association more important then the freedom from being harmed by large associations with amoral machinations. The original argument asks that if large corporations inherently obscure moral outcomes, maybe they are immoral, which is an argument that puts these two moral axioms in conflic…

Morality is highly variable, depending on the observers beif system. Legality is the only framework that we can establish in common. Ethics comes in second as it can be established by a group and does not bind those outside the group.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#498

Post by coding123 »

eat_veggies wrote: post #481
coding123 wrote: post #475

Earlier quoted context omitted.

But what is harmful software? Today, while there are arguments about facebook being too addictive, it's really just competition between platforms, it's not everyone's opinion that the software engineers at facebook should be liable at this point for making an addictive algorithm. At some uncertain point in time, there's the possibility that facebook will be sued for the addictiveness and lose. Should that happen, I d…

If Facebook gets sued for its addictiveness and loses, then that would be a pretty solid judgment of its harm, wouldn't it? And it'd be up to the court to decide whether the engineers (and which ones) were complicit. There is room for nuance here. > I don't see why the software engineers should be partially to blame. Because they chose to build it! Software engineers aren't ignorant and helpless executors of instruct…

That makes no sense. That's the exact angle I'm trying to argue isn't feasible. It should absolutely be the corporate entity that takes the blame for things like that. If we were to change things to make the engineers directly responsible, we're just adding an entire industry that's not necessary.

We'll end up creating an entire class of insurance for software development liability. It doesn't really change anything except again, raise the cost and shift burdens.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#499

Post by coding123 »

doorstar wrote: post #487
eat_veggies wrote: post #460

Earlier quoted context omitted.

There are not two sides, because both management and engineering should face consequences for building harmful software. And I'm not sure it's appropriate to test these "what ifs" as if looking for logical flaws in such a system, because a well-functioning court and jury should easily be able answer these questions, and I'm sure you had the "correct" answers in mind when you wrote them. Whether or not our courts and…

> a well-functioning court and jury should easily be able answer these questions, I'm going to ignore the qualifier of 'well functioning' which obviously up for debate. The process of being charged with a crime, being put on trial, wondering at the consequences of the outcome etc., is no joke. It is a tremendously time-consuming, expensive, and stressful processes and even if you are acquitted there is no undoing the…

That's exactly it. It also begs the question, if corporations already have liability insurance, what's the point of forcing all of its employees to get similar coverage. It really is just giving more money to the insurance industry.

Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers

#500

Post by hosh »

qayxc wrote: post #337
hosh wrote: post #245

I think I prefer the three permaculture ethical principles. They are much more condensed, more broadly applicable to any technology at any tech level, and much more holistic. These are: 1. Care of Earth 2. Care of People 3. Fair Share They can be applied to technology as part of the whole ecology, not just human-centric. It puts life and death in the proper context of the whole, rather than the way modernity had twis…

Caring about ecology is very much human-centric. Neither the planet nor its biological inhabitants "care" about anything. They just are (as in they exist). The main driver for protecting the ecosystem is the fact that we - as a species - can't survive (for long) without it. Granted, there are some who strongly believe that becoming a race of cave dwelling mole people or inhabitants of glorified snow globes in an inho…

Care about the ecology is not just about the environment as if it were something separate, or abstracted. There is a very practical, pragmatic reason not to sh$t where you eat. At least, not without composting with charcoal, or using one of those ecovats to clean it first, before feeding it back to the garden.

As far as whether the planet or biology “care”, we will have to disagree and leave it at that. While I think the word “care” is overloaded with a number of meanings, many are human-centric, I do have beliefs that there is a planetary and plant intelligence. I don’t think the Universe is as uncaring as you think, though there are many, many unpleasant things.

Post reply on HN