Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tell HN: Duty of care

news.ycombinator.com

Post reply on HN
Print view View original

21–30 of 55 posts

Re: Tell HN: Duty of care

#21

Post by hn_throwaway_99 »

SentinelLdnma wrote: post #16
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

I took an Ethics for Engineers course as an elective in college (sometime during the stone age). Wrote a paper on Therac-25. Had I not, I may not even been cognizant of the risk involved. A single week lecture on this topic could move the needle. Or regulators like FDA could demand to see corporate training materials given to software "engineers" (I concur on the quotes) on how to promote product safety.

Agreed and thanks! I actually didn't realize that tweet reply to your original reply was also from you - it's what got me thinking about clear engineering codes of conduct in the first place.

Re: Tell HN: Duty of care

#22

Post by lolinder »

clnq wrote: post #18
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

The labor pool of software engineers is very diverse in terms of backbones, ethics and morals. It puts replaceable engineers in an awkward spot to object to unprofessional requests. For an anecdotal example, I have been told that standing up for ethics in software engineering would have negative consequences on my performance (implied: bonus, promotions, career). I have left that company, and they probably found some…

What happens in other engineering disciplines is the government revokes your license if you make a decision that violates your discipline's standards. Companies can't generally just force their engineers to do irresponsible things because even if they fired you and hired someone new, that new person would be putting their own future employability on the line by conceding. Better to get fired from a bad firm than to have to find a new career.

Obviously this doesn't solve all the problems, but it works as well as any solution I can think of.

The caveat when it comes to software is that coming to a consensus of what the standard procedures and policies should be would be nearly impossible. If and when software joins the licensed engineering fields, a lot of people are going to be very upset at whatever the requirements end up being.

Re: Tell HN: Duty of care

#23

Post by SentinelLdnma »

clnq wrote: post #18
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

The labor pool of software engineers is very diverse in terms of backbones, ethics and morals. It puts replaceable engineers in an awkward spot to object to unprofessional requests. For an anecdotal example, I have been told that standing up for ethics in software engineering would have negative consequences on my performance (implied: bonus, promotions, career). I have left that company, and they probably found some…

As others have noted, professional licensure (which can be revoked) is used in other industries.

Not feasible to ask this for every developer, but for safety critical systems it should be mandatory.

Re: Tell HN: Duty of care

#24

Post by SentinelLdnma »

trinovantes wrote: post #20
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

In some jurisdictions, software engineer is a protected job title that requires professional licensing (ever wonder why Google only hires "developer" in their Waterloo office but "engineer" elsewhere?). IIRC the engineer that wrote the code for the Volkswagen scandal was jailed for failing his professional obligations. For OP's case case, I'd assume/hope a professional engineer would need to sign off on this change o…

Hardware came from oversees and there was no licensed PE on the US side. But you're right, there should have been.

This was before the device was submitted to the FDA for approval. Chance they would have rejected it. A fallback position for a developer not willing to Play It Hard would be to make sure the risk docs submitted to FDA document this failure path. There is a whole process you are supposed to go through to analyze risks / harms and whether your controls are adequate.

Re: Tell HN: Duty of care

#25

Post by hn_throwaway_99 »

clnq wrote: post #18
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

The labor pool of software engineers is very diverse in terms of backbones, ethics and morals. It puts replaceable engineers in an awkward spot to object to unprofessional requests. For an anecdotal example, I have been told that standing up for ethics in software engineering would have negative consequences on my performance (implied: bonus, promotions, career). I have left that company, and they probably found some…

> The labor pool of software engineers is very diverse in terms of backbones, ethics and morals.

This is exactly my point. The same is true for other engineering disciplines, too, but by codifying their ethical responsibilities (and, in some cases, assigning liability to engineers who forego those responsibilities), other professional engineering organizations help to ensure a bare minimum for what is and isn't allowed in their profession.

I'll give another example that has much less dire consequences than the OP's. 90% of "scarcity marketing", e.g. "Act now! There are only 2 rooms left!" or even "8 other people are looking at this property!", is complete and total bullshit. I've even seen A/B tests where they developers were like "yeah, the data here is not real, we just want to see if it has an effect." Why is this even in the realm of acceptability? There is no gray area here - it's not just a "dark pattern". It is 100% outright lying. Yet I never heard someone stand up in those product reviews (myself included, so I'm no hero either) and say "How can we spend so much time on our 'company values' when this is obviously bullshit and slimy?"

I wish there were a "software engineering code of conduct" that said that outright lying to end users is verboten, and that software developers can be held personally liable if they are aware of the lie and still implement it.

Re: Tell HN: Duty of care

#26

Post by clnq »

lolinder wrote: post #22
clnq wrote: post #18

Earlier quoted context omitted.

The labor pool of software engineers is very diverse in terms of backbones, ethics and morals. It puts replaceable engineers in an awkward spot to object to unprofessional requests. For an anecdotal example, I have been told that standing up for ethics in software engineering would have negative consequences on my performance (implied: bonus, promotions, career). I have left that company, and they probably found some…

What happens in other engineering disciplines is the government revokes your license if you make a decision that violates your discipline's standards. Companies can't generally just force their engineers to do irresponsible things because even if they fired you and hired someone new, that new person would be putting their own future employability on the line by conceding. Better to get fired from a bad firm than to h…

The caveat is very true, and I think that I would be unhappy if a licensing body told me I'm suddenly "not qualified" to do the work I've been doing well for many years. Besides, it wouldn't be just for me to bear licensure costs (whether direct or indirect by being out of work while I get licensed) to correct an ethical fault in companies.

Perhaps licensure could be an effective solution, but one that is not very empathetic to engineers. Maybe some kind of a government-owned ethics controller/body to handle unethical software would be more just for engineers. Although it could also be very inefficient.

Re: Tell HN: Duty of care

#27

Post by bo1024 »

lolinder wrote: post #19
SentinelLdnma wrote: post #16

Earlier quoted context omitted.

I took an Ethics for Engineers course as an elective in college (sometime during the stone age). Wrote a paper on Therac-25. Had I not, I may not even been cognizant of the risk involved. A single week lecture on this topic could move the needle. Or regulators like FDA could demand to see corporate training materials given to software "engineers" (I concur on the quotes) on how to promote product safety.

My school required a software-focused ethics course in order to graduate with a CS degree. It was great—one of the few courses that I lean on on a regular basis in my work. Good on you for taking it as an elective, but it's weird to me that any degree in any topic can be accredited without having a mandatory ethics course, let alone a degree in a science/engineering field.

Would love to hear some specifics from the course that come up for you at work!

Re: Tell HN: Duty of care

#28

Post by mmusc »

hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

Back when I was studying Computer Science, the fact that Computer engineering is not engineering was a very hot topic. Engineering comes with certain guarantees. If a you don't put a load more than X on a bridge for the next X years. It will not break.

How can we offer that guarantee as a profession when hardware, drivers, OSes and even libraries that we use change and shift without our control and sometimes consent?

Re: Tell HN: Duty of care

#29

Post by Natsu »

SentinelLdnma wrote: post #16
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

I took an Ethics for Engineers course as an elective in college (sometime during the stone age). Wrote a paper on Therac-25. Had I not, I may not even been cognizant of the risk involved. A single week lecture on this topic could move the needle. Or regulators like FDA could demand to see corporate training materials given to software "engineers" (I concur on the quotes) on how to promote product safety.

I sat through a very similar class once upon a time.

Re: Tell HN: Duty of care

#30

Post by SentinelLdnma »

mmusc wrote: post #28
hn_throwaway_99 wrote: post #13

I think the fact that software "engineering" (and, yes, those are meant to be scare quotes) doesn't have the same level of rigor of other engineering disciplines is what is truly at fault here. Most other professional engineering disciplines have clear, codified rules of ethics. If the Chairman of a public company told a building engineer "Can you get rid of a few of these bolts on this beam here? I don't like the wa…

Back when I was studying Computer Science, the fact that Computer engineering is not engineering was a very hot topic. Engineering comes with certain guarantees. If a you don't put a load more than X on a bridge for the next X years. It will not break. How can we offer that guarantee as a profession when hardware, drivers, OSes and even libraries that we use change and shift without our control and sometimes consent?

Software "engineering" will definitely need to be looked at differently from physical, but there are still things we can do.

Critical systems (or at least the critical subsystems) must not allow any unapproved/untested changes. Hence why it's better for safety controls to be hardware/firmware and not part of a general-purpose OS.

It's one reason why you see separate payment terminal hardware on self-checkout kiosks. The payment hardware is more tightly controlled whereas they can modify the kiosk much faster.

There are also RTOS (real time OS) that offer execution time guarantees. Used in aerospace.

Post reply on HN