Brad Smith in 2007: Linux violates 235 MS patents, and the FOSS community must pay up. https://web.archive.org/web/20070514143032/https://money.cnn...
Brad Smith in 2020: We Need a Hippocratic Oath for Software Engineers.
Microsoft President: We Need a Hippocratic Oath for Software Engineers
411–420 of 544 posts
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#412Earlier quoted context omitted.
> accept risks from intentionally violating the spirit of that limitation. What risk? Since it's the engineers, not the executives that take any oath of professional conduct, there wouldn't be any risk for any executive. All this would get us is a legal framework for throwing engineers under the bus via an ethical commission if they do something silly like blowing the whistle on an unethical decision higher up in the…
> there wouldn't be any risk for any executive This isn't a real problem. I take it you have never worked outside of software. You don't need a law license to be the business manager of a major law firm or a medical license to manage a large hospital network. How often does such problems occur in major law firms or large hospital networks? In other words this is a hypothetical complaint of some future condition that…
> I take it you have never worked outside of software.
You take it wrong, and reasoning by analogy with other fields is a very risky line of logic, because they're not at all similar in terms of internal regulation and professional autonomy.
First of all, in fields like medicine, that kind of responsibility is necessary because clinical autonomy is far wider than any kind of professional autonomy that software engineers have, at any level. And it works precisely because this autonomy doesn't come only with the responsibility of adequate care, it also comes with the ability to say "no" without fearing for your employment or license. Physicians enjoy a great deal (if faltering, and arguably still insufficient!) of protection for defending the interests of their patients, and firing someone for it is a pretty risky and expensive affair. Software engineers enjoy pretty much zero protection in this regard.
If you want engineers to bear that kind of responsibility, that's great, and I'm all for it, too. But in order for it to have any use, you also have to give them the means to defend the standards of professional conduct they must now conform to. Are you also prepared to:
* Defer to an external body of experts to decide if a technology is fit for commercial use or not?
* Forbid an engineer from arbitrarily overriding another engineer's bugfix?
* Have product managers defer to an ethics commission, ran by engineers and engineering ethics specialists, when deciding what features to prioritize or introduce? And if the ethics commission says "no" to a feature, then it's a "no" that not even the board or the executive team can override?
* Make it illegal to ask engineers to introduce a workaround with known flaws, rather than the proper fix, without explicitly informing customers about the known flaws?
* Make it illegal to fire engineers for not executing a manager's order if that order violates the engineers' standards of conduct?
* Defer to an external engineering organization to decide who even gets to call themselves a "software engineer"?
Second, one of the reasons why managing large hospital networks by people without a medical license works (more or less...) is that, for example, there's usually no legal way for the CEO to walk into an OR while cardiac surgery is happening and request the surgeon unplug the CPB machine, even if that means killing the patient. Even if there's no internal regulation being broken, criminal law is more than enough to keep that sort of thing in check. There's hardly anything that would currently prevent the CEO of a software company from walking into an office and requesting that an encryption feature be backdoored, for example.
Without the kind of autonomy and legal protection that would enable software engineers to defend such an oath, its provisions would be completely meaningless. What kind of benefits do you think the software industry, and society in general, would get from an oath that's more risky to defend than to break?
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#413Earlier quoted context omitted.
this is precisely the problem. few physicists knew they were all collectively building a nuclear bomb because the big picture was well compartmentalized. feynman only found out because he picked the locks of his colleagues to piece it all together.
Can you point to the part of the book where it is stated that only few physicist (not including Feynman) knew this? I remember it very differently. From my memory: - lock picking was for fun and didnt gain info - some warehouse workers didnt know about critical mass of uranium and stored the stuff to closely together
ah maybe you're right. memory might be foggy. time for a fresh re-read!
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#414I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…
There is the problem that everything can be subdivided in a bunch of menial task so every cog in the system isn't aware of their impact. It already happens a lot in software.
the idea of Hippocratic Oath reminds me of Asimov's Three Laws of Robotics in "The Naked sun" (SPOILERS ahead): the detective realises that the normally quoted First Law of Robotics ("A robot may not injure a human being or, through inaction, allow a human being to come to harm.") is actually just an approximation, he argues that the real Law is "A robot may do nothing that, TO ITS KNOWLEDGE, will harm a human being; nor, through inaction, KNOWINGLY allow a human being to come to harm."
This is important because even though robots really try their best, different robots could perform sub-tasks that look very harmless by themselves, but combined kill a human being:
- A robot is instructed to pour this bottle of poison into a caraffe of water and then leave the room
- Another robot is instructed to enter the room, take the caraffe of water and give it to a human to drink
The human is poisoned, but none of the robots are directly responsible (in the first law sense). Is the act of connecting the two dots the evil deed.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#415Earlier quoted context omitted.
I can’t imagine software engineers staying in a company long enough to acquire it. For this to work, software engineer compensation would need to change radically.
Maybe that's the point of this oath? Software engineers will refuse to work on these areas, unless demands of the oath can be met.
A doctor refusing to do a procedure because he worries for his patient is seen as a good guy doing what is right, he is in his opinion saving a life. In addition he is trying to avoid the massive cost of a medical malpractice suit.
A software developer refusing a job because it does not meet his ethical parameters is just an unemployed software engineer.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#416Software engineers usually aren't in charge of the features they write. They can refuse to do their job at their own peril. Maybe we need a hippocratic oath for product managers or something.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#417Earlier quoted context omitted.
From my limited information the MCAS code was primarily causing problems in association of incorrect readings by damaged sensors. Of course one could argue that this is an engineering failure because the MCAS failed to account for wrong sensor input but when you consider the legal implications of a MCAS fallback there is actually not much that can be done on the software side. The MCAS is an optional component that r…
> he works at wants to use the MCAS for a non ethical purpose (namely operate and hide the existence of MCAS even when it is unsafe to do so). You think a programmer, handed a spec and asked to implement it, can be expected to know that their employer (or the employer's customer) wants to use it for a "non ethical purpose"? Again, I can't know for sure, but I doubt the programmers who wrote MCAS (who most likely didn…
With all of that being said, were these software engineers (probably subcontractors) even given access to actual MCAS readouts or, more likely, virtualizations of expected readouts. These people probably didn't account for this type of faulty readout because the virtual machine never put out that type of fault.
Most types of development in these large companies is so compartmentalized that it's next to impossible to see the whole structure from a software engineers prospective. You need to be at a management level to understand how most of the pieces really come together, which is the only place where one of these "oaths" might make an influence. At that point, however, the selection is so goal oriented, I have a doubt as to whether or not people would take that oath.
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#418https://en.wikipedia.org/wiki/Iron_Ring
>The Iron Ring is a ring worn by many Canadian-trained engineers, as a symbol and reminder of the obligations and ethics associated with their profession. The ring is presented to engineering graduates in a private ceremony known as the Ritual of the Calling of an Engineer.[1][2] The concept of the ritual and its Iron Rings originated from H. E. T. Haultain in 1922, with assistance from Rudyard Kipling, who crafted the ritual at Haultain's request.[1][3]
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#419I’m not a fan of comparisons to the Hippocratic oath. The greatest risk to AI ethics is not the ethics of software engineers but the ethics of the software engineering process. By the time tasks are handed to engineers, most of the ethical decisions have been made by product managers, designers, and business stakeholders who are focused on their own goals. Software engineers are accountable to their bosses before the…
This thread is littered with this attitude, that the developers don't actually have a seat at the table where "what to build" decisions are made. This was not always the case. How the hell did we let that happened?
Here's a prospective from someone who became a software dev in my late 20s after other jobs. Relatively speaking, if you can write working code and show up on time, you have a lot of leverage in getting and keeping employment, more than most other middle to upper middle class professions. This means org politics has less effect on you, and less incentive to get involved.
As a dev you can, but don't have to, think as much about the politics and operations of your org for all kinds of reasons. You are relatively harder to replace so internal politics tends to matter less, and if the org makes decisions you don't like, you can be confident that you can leave and find something else versus the long and often unfruitful process of trying to change an org from within.
Politics can and often is messy, how often have you heard something like "I just want to build things" (it's how I feel for sure), if you can get paid well to do that, why get involved with a messy decision making process?
Re: Microsoft President: We Need a Hippocratic Oath for Software Engineers
#420IMO having a hippocratic oath for software engineers is entirely pointless. There are plenty of people who would say the words with no intention of living up to them - not everyone is going to share your values. I can't say for sure, but I have a feeling that the hippocratic oath is not what's stopping an unethical doctor from behaving unethically. Instead, it's the risk of lawsuits and loss of their license to pract…
For sure, I am one of these people you mention. I feel very little to no loyalty to any company I have ever worked for. The reason is that they can and would fire me at any time if the conditions where right.
Hence, I am ready to move to another position at any time if the conditions are right (pay, benefits etc). If they want me to take an oath I would probably not take the job today but if they would pay me a ridiculous amount for example, I would most likely take the job and just say the oath or whatever you have to do and just never care about it again.
Just because I am forced to say something doesn't mean I believe it. Maybe it's because I am a natural rebel that hates to be told things, despises "mission statements" or "company values" but I feel like there is a lot of people like me.