Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a Butlerian Jihad against AI

erikhoel.substack.com

Post reply on HN
Print view View original

41–50 of 161 posts

Re: We need a Butlerian Jihad against AI

#41

Post by toolz »

> And some things are abominations, by the way. That’s a legitimate and utterly necessary category. It’s not just religious language, nor is it alarmism or fundamentalism. The international community agrees that human/animal hybrids are abominations—we shouldn’t make them to preserve the dignity of the human, despite their creation being well within our scientific capability.

This is nothing more than an appeal to authority, no? Even in this proposed axiom there's plenty of room to disagree (even if the author rejects that there is)

Re: We need a Butlerian Jihad against AI

#42

Post by Manuel_D »

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's more rooted in fiction than engineering.

It's interesting to ponder how we should go about building and interacting with "strong AI", and questioning whether we should even build it in the first place. But I really don't think any detailed moral frameworks can be built when we have no real idea of what a "strong AI" would look like.

Also, it's worth reminding people that in the Dune universe the Butlerian Jihad led to millennia of stagnation and control of society by a narrow elite: The Spacing Guild, the Bene Gesserit, and the Landsraad.

Re: We need a Butlerian Jihad against AI

#43

Post by bpodgursky »

dry_soup wrote: post #39

It says a lot about what injustices we have learned to accept that AI alarmists focus almost exclusively on the scifi-level hypothetical dangers of AI, rather than the very real problems it already causes today. Those problems largely fall into three categories that I can think of off the top of my head at 1am: 1. AI is a convenient way to justify potentially uncomfortable decisions you would have made otherwise (idl…

"AI alarmists" are worried because the worst-case outcomes of AGI are mistakes you cannot ever fix.

All the rest of these are bad, but they are problems we can fix given time and thought, because we will still exist. Extinction-level events decrease all future human utility to zero, and so should be treated with extraordinary care.

Re: We need a Butlerian Jihad against AI

#44

Post by anigbrowl »

AndrewKemendo wrote: post #9

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

imho precision is a chimera which often leads to an excess of certitude; acceptance and awareness of uncertainty often leads to better decision-making.

Put another way, a laser pointer is not a very good tool with which to explore a cave, unless you can systematically measure it over the whole cave, an expensive and time-consuming process. If you're exploring a new cave, you might be better off with weak omnidirectional illumination like a lamp.

Re: We need a Butlerian Jihad against AI

#45

Post by mandelbrotwurst »

bookofsand wrote: post #37
vngzs wrote: post #12

Earlier quoted context omitted.

I do believe it's reasonable to draw a distinction between what we're doing now (which is essentially just "statistics") and what Musk warns about. It's a mistake to believe strong AI will just be a more powerful iteration of today's weak AI. He is arguing to slow developments toward general intelligence, not developments in any narrow field.

With high probability, 'just statistics' is an essential component of strong AI. Another essential component is embodiment, of which self-driving cars, and also military drones, are canonical examples. Researchers are taking the correct essential steps towards strong AI, it's a matter of (short) time until they succeed.

What are the requisite “correct essential steps” and how did you determine that they will necessarily lead to an AGI?

Re: We need a Butlerian Jihad against AI

#46

Post by pshc »

Dylan16807 wrote: post #35
ampdepolymerase wrote: post #7

> To slim results. Elon Musk explained his new nihilism about the possibility of stopping AI advancement on the Joe Rogan podcast, when he said: “I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened.” Rather rich coming from one of the self driving car market leaders. It certainly makes business sense to mislead academia and the po…

> Rather rich coming from one of the self driving car market leaders. You don't need anything close to strong AI to do a reasonable job of driving a car. It seems like 90% of the problem is object recognition, and even in terms of brain-equivalent logic that's a really low bar.

It’s the 10% that gets you. To safely operate a car in all reasonable situations might very well require Strong AI. The car needs to be able to problem solve and make inferences about road conditions up ahead.

Re: We need a Butlerian Jihad against AI

#47

Post by ampdepolymerase »

sammalloy wrote: post #16

I think it would be easier to emphasize and teach philosophy and ethics of computer science. The problem is, they can't even do this for business students, so focusing on the existential risk of AI alone is a drop of rain in the ocean. If you follow the topic, of let's say, sustainability, then you know this is a huge blind spot in economics and business. Look at only palm oil production as an example. The current th…

Because ethics in engineering is mostly about codified rule compliance rather than the deep navel gazing taken by actual philosophy students. The latter also rarely yields useful answers. Crack open one of the "professional" engineering ethics guide books and 90% of it is thou shalt not build a bad bridge/engine/circuit because it is very bad and you should report your boss to the authorities if they do. I never understood the moral uppity and delusion those engineers have. If a bridge falls it is bad, unless if it is explicitly designed to kill enemy soldiers, then it is good. You can extend this to drones and enemy schoolbuses and the entire defence industry if you want. The so-called "engineering ethics" field should rebrand and follow the financial industry. You follow guidelines because the compliance department demands it to cover the company's derriere. If you don't, your company will get fined. Skip the self-righteous morality because its only purpose is to reduce the principle agent problem for the managerial and asset owning class.

Re: We need a Butlerian Jihad against AI

#49

Post by anigbrowl »

erikhoel wrote: post #27
AndrewKemendo wrote: post #9

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

There are all sorts of ways to build intelligences. Humans are unique in that they are mammals (defined by having mothers). Mothers raise us with love, and teach us, for our helpless first years. We also have to act in communities. So there is a sense in that we are very lucky - in humans, our intelligence correlates with our altruism. In the grand space of possible minds, it is very unlikely that altruism and morali…

Tiny nitpick, but quite a lot of birds are nurturing despite not being mammals. You can find altruistic (or at least mutualist) behaviors in many other taxa.

Re: We need a Butlerian Jihad against AI

#50

Post by elurg »

ampdepolymerase wrote: post #7

> To slim results. Elon Musk explained his new nihilism about the possibility of stopping AI advancement on the Joe Rogan podcast, when he said: “I tried to convince people to slow down. Slow down AI. To regulate AI. This was futile. I tried for years. Nobody listened. Nobody listened.” Rather rich coming from one of the self driving car market leaders. It certainly makes business sense to mislead academia and the po…

* AI research already has high barriers to entry because the required computing resources are expensive.

* Self-driving AI will be heavily regulated for reasons not related to other AI regulation.

I don't see how more AI regulation would financially help Tesla or Elon Musk. In fact "more regulation of self-driving" is something that many Tesla competitors and critics support.

Post reply on HN