Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Not Build the Torment Nexus

buttondown.com

Post reply on HN
Print view View original

11–20 of 42 posts

Re: How to Not Build the Torment Nexus

#11

Post by Apreche »

yummypaint wrote: post #7

It's pretty wild that software is the only "engineering" discipline without any concept of professional ethics or loyalty to human safety that superceeds the whims of the employer. Why do people think that is? Have there been any attempts to change this from the inside over the past decade? Where are professional associations like the ACM in all of this? It's a shameful state of affairs and reflects poorly on the who…

When I was in college majoring in CS in the early ’00s I took several classes about ethics and technology. I also took plenty of liberal arts and literature classes throughout my entire education, as well as reading many sci-fi novels for pleasure. I learned the lesson of not building the torment nexus early and often. Thus far I have refused employment at any places doing so. I have an ethical and moral code that comes before anything an employer asks of me.

I think the problem is that clearly not everyone took the same path, and did not get the same moral education. I’m not religious at all, but religions do play a part in morally educating people. Not only are fewer people religious, but more of the religions that are popular are outright immoral in their teachings.

For profit companies do not act morally when not required to by law. Remember when Google hired some AI ethicists, and then fired them? Heck, they couldn’t even maintain their elementary school ethical code of “don’t be evil.” Companies who engineer bridges, medical devices, aircraft, etc. are regulated by law. They don’t follow ethical engineering practices of their own volition. They do it only because we force their hand.

We’re starting to see now what it looks like to regulate tech with some of the policies in the EU. While those regulations are flawed in several ways, they are also working properly. GDPR does have positive impact on privacy. Digital Markets Act is forcing Apple to allow side-loading.

We can defeat the torment nexus by simply outlawing it.

Re: How to Not Build the Torment Nexus

#12

Post by DicIfTEx »

kelseyfrog wrote: post #2

If you build the Torment Nexus, it might even be ever so slightly less tormentful[1] than it otherwise could have been because you're a good person, perhaps even slightly better than most. This thing you eventually come to believe, of course, is exactly what one must believe for the Torment Nexus to be made. Sure, there are structural and economic incentives that tilt the playing field, but this is the belief that is…

Hannah Arendt considered this argument in her essay "Personal Responsibility Under Dictatorship":

> In the tumultuous discussion of moral issues which has been going on since the defeat of Nazi Germany, and the disclosure of the total complicity in crimes of all ranks of official society, that is, of the total collapse of normal moral standards, the following argument has been raised in endless variations: We who appear guilty today are in fact those who stayed on the job in order to prevent worse things from happening; only those who remained inside had a chance to mitigate things and to help at least some people; we gave the devil his due without selling our soul to him, whereas those who did nothing shirked all responsibilities and thought only of themselves, of the salvation of their precious souls… (p 34)

But, ultimately, she finds the excuse lacking:

> Politically, the weakness of the argument has always been that those who choose the lesser evil forget very quickly that they chose evil. (p 36)

https://grattoncourses.wordpress.com/wp-content/uploads/2016...

Re: How to Not Build the Torment Nexus

#13

Post by salawat »

yummypaint wrote: post #7

It's pretty wild that software is the only "engineering" discipline without any concept of professional ethics or loyalty to human safety that superceeds the whims of the employer. Why do people think that is? Have there been any attempts to change this from the inside over the past decade? Where are professional associations like the ACM in all of this? It's a shameful state of affairs and reflects poorly on the who…

My ethics curriculum while getting my CS degree basically highlights the last decade+ of big name tech companies as "things that should not be done for obvious reasons". So I avoided all of them, looking for some of the most boring, banal tech jobs/applications of computers to clerical work imaginable.

Imagine my surprise when even most of those jobs were founded on things/goals/ambitions/central risks that were also profound ethical lapses. I've gotten to the point where I'm honestly wondering whether humanity is even ready for ubiquitous computing, to which seemingly the answer lately trends toward "hell no". Great thing to only realize after 10 years in the industry.

Point being, ethics is something self enforced, and we've taken great pains to ensure there is no professional licensure body or anything else around software; in particular because of the fundamental asymmetry created by locking that fire away from mortals behind a bunch of barriers to entry. Those with tech will have an advantage over those without. This is a certainty. The price of that (intentionally opening access to expertise for anyone curious) is what we have now. The tech that gets implemented is a reflection of the collective soul of humanity. If Greed is God, and the Deadline and Sale take priority over Safety, Fitness for Use, Quality, and Characterization of the System-Under-Scrutiny; then previously mentioned lapses in the ethicality of implementations of computer systems are what we're gonna get, and the conscientious objector will just be walked out the door and the next practitioner less afflicted by scruples will be walked in instead.

I'm open, as I've always been to putting a thumb on the scales through greater organization of active practitioners to actually make a means to ensure some subset of systems don't get implemented, but I'm not sure that's the right answer. The right answer is to improve ourselves and our non-computerized ways of life so there is no damn incentive to make the Torment Nexus.

Easy to say right? But that execution... Oofta.

Re: How to Not Build the Torment Nexus

#14

Post by blamestross »

yummypaint wrote: post #7

It's pretty wild that software is the only "engineering" discipline without any concept of professional ethics or loyalty to human safety that superceeds the whims of the employer. Why do people think that is? Have there been any attempts to change this from the inside over the past decade? Where are professional associations like the ACM in all of this? It's a shameful state of affairs and reflects poorly on the who…

I think we need our own variant of the "iron ring" at this point. I wear a similar steel ring on my pinky as an oath reminder.

Its a lot of work to grass-roots something like that, and I don't have the charisma for it.

Re: How to Not Build the Torment Nexus

#15

Post by shortrounddev2 »

yummypaint wrote: post #7

It's pretty wild that software is the only "engineering" discipline without any concept of professional ethics or loyalty to human safety that superceeds the whims of the employer. Why do people think that is? Have there been any attempts to change this from the inside over the past decade? Where are professional associations like the ACM in all of this? It's a shameful state of affairs and reflects poorly on the who…

I think other engineering disciplines grew up in an era where workers rights and public safety was becoming a big focus of government. Software engineering grew up in a time where we took government for granted; even demonized it

Re: How to Not Build the Torment Nexus

#16

Post by thecosas »

laughingcurve wrote: post #9
throwaway404040 wrote: post #4

Earlier quoted context omitted.

It's telling how little discussion this post is generating on HN. Perhaps a little too uncomfortable for a large portion of the crowd?

Usually when crazy people get on the metro and start yelling their political opinions at no one, we all know it’s best to ignore them.

Perhaps those of us voicing no opinion at all are the crazy ones.

Re: How to Not Build the Torment Nexus

#18

Post by Havoc »

Reminds me of the feelings combat drone remote pilots speak of where they’re also often forced into difficult ethical positions. It’s literally the job but there is clearly also harm being done. A terrorist than turns out to be not a terrorist etc

Re: How to Not Build the Torment Nexus

#19

Post by falcor84 »

> Tech was a sector that made us think of humankind moving forward, possibly into some happy Star Trek like future where no one needed money and pie magically appeared in your wall if you said “Magic wall, pie me!”

Am I too naive in thinking that the AIs we're working on are a prerequisite step on the way towards Star Trek's post-scarcity future?

Re: How to Not Build the Torment Nexus

#20

Post by dweinus »

falcor84 wrote: post #19

> Tech was a sector that made us think of humankind moving forward, possibly into some happy Star Trek like future where no one needed money and pie magically appeared in your wall if you said “Magic wall, pie me!” Am I too naive in thinking that the AIs we're working on are a prerequisite step on the way towards Star Trek's post-scarcity future?

No, whether this tech could lead to a post-scarcity future is debatable. The naive part is thinking that it will lead to it, given the way we use it and the power structures that profit from it.
Post reply on HN