Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We need a Butlerian Jihad against AI

erikhoel.substack.com

Post reply on HN
Print view View original

101–110 of 161 posts

Re: We need a Butlerian Jihad against AI

#101

Post by jaked122 »

Radim wrote: post #88
BitwiseFool wrote: post #32

I like to think of AI-Genesis through the lens of what humanity has already done through domestication. We take something primitive and progressively adapt it to serve a greater utility. I think working dogs are the most interesting example of this. We've taken a species, the wolf, and made it smarter while also making it want to do work, learn tricks, and follow orders. Of course, you still need to train the animal…

> We've taken a species, the wolf, and made it smarter In what universe is a dog smarter than a wolf? I think you got the correlation between animal domestication and their intelligence backward :)

Yeah, but have you tried getting a wolf to guard a sheep?

\They're nowhere near as clever at that

Re: We need a Butlerian Jihad against AI

#103

Post by neltnerb »

Tenoke wrote: post #87
Manuel_D wrote: post #42

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…

I dont really get how people see the progress of the last 10 or even 5 years and then imply that things won't improve all that much further than that for a whole generation.

The oldest publication from DeepMind is 2011, just for a post in the ground. Yes, I know neural networks are older than that.

It would be a little disappointing if we could even guess how an AI system in 2040 would work, I don't think many people in 2000 would have guessed neural networks would be the most useful type in regular use in 2020.

In my field of expertise there are grad students today taking classes on topics that when I finished my PhD in 2010 no one had heard of yet. Literal cases of people finding closed form solutions for equations that we had to solve with approximation before.

Re: We need a Butlerian Jihad against AI

#104

Post by AndrewKemendo »

tick_tock_tick wrote: post #68
AndrewKemendo wrote: post #9

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

> extinct eventually and should think about making a better successor for the human species Citation needed. As it currently stands it seems incredibly unlikely we won't expand to most of our local group making extinction incredible unlikely.

You're suggesting that humanity in it's current form could survive the expansion of the sun and make it to the eventual heat death of the universe?

Re: We need a Butlerian Jihad against AI

#105

Post by AndrewKemendo »

erikhoel wrote: post #27
AndrewKemendo wrote: post #9

We should be actively building this new "AI Species" because we are going to be extinct eventually and should think about making a better successor for the human species. The morality argument is nonsense. How about this: "The primary objective of humanity should be to build an intelligent system with far more precise perception, reasoning and physical manipulation capabilities than humans" That's my starting point.

There are all sorts of ways to build intelligences. Humans are unique in that they are mammals (defined by having mothers). Mothers raise us with love, and teach us, for our helpless first years. We also have to act in communities. So there is a sense in that we are very lucky - in humans, our intelligence correlates with our altruism. In the grand space of possible minds, it is very unlikely that altruism and morali…

None of that is unique to humans. Plenty of other primates and fowl have similar systems.

Re: We need a Butlerian Jihad against AI

#106

Post by guscost »

Tenoke wrote: post #87
Manuel_D wrote: post #42

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…

I dont really get how people see the progress of the last 10 or even 5 years and then imply that things won't improve all that much further than that for a whole generation.

You could have said the same exact thing about aerospace in 1970.

Re: We need a Butlerian Jihad against AI

#107

Post by layfellow »

jrochkind1 wrote: post #79
at_a_remove wrote: post #11

One of the problems the Butlerian Jihad ran up against, aside from the inevitable skirting of the lines from Richese and Ix (many machines on Ix, new machines), is that it runs directly counter to "Thou shalt not disfigure the soul." Replacement of AI with Mentats (as well as other narrow specialities) has done nothing but disfigure the soul. We see few Mentats -- aside from Paul and eventually another -- who are not…

If there is literally no possible future with dignity for all consciousnesses... that'd be pretty depressing.

Ian M. Bank's The Culture universe is such a future, which I think is attainble in a few millennia.

Powerful super-human AI minds will eventually and inevitably take over; which I think is fine provided they recognize all sentient beings as equals.

Re: We need a Butlerian Jihad against AI

#108

Post by plorkyeran »

Tenoke wrote: post #87
Manuel_D wrote: post #42

I really doubt we will have the capability of building "a machine in the likeness of a human mind" in my lifetime. Present AI systems are essentially just function fitting. Building big probabilistic systems that we optimize with loads of training data. This is a far, far cry from the "strong AI" that people are so afraid of. I really think that people writing these sorts of pieces have an understanding of AI that's…

I dont really get how people see the progress of the last 10 or even 5 years and then imply that things won't improve all that much further than that for a whole generation.

Most areas of study up being S-curves, not exponential gains forever.

Re: We need a Butlerian Jihad against AI

#109

Post by bpodgursky »

killingtime74 wrote: post #69
bpodgursky wrote: post #25

Earlier quoted context omitted.

That's exactly my point. I did, explicitly, say it was unpleasant, and something you should only do in extremis -- when you genuinely believe that extinction a likely alternative. But the US could absolutely glass every semiconductor fab (and datacenter, and research facility) in mainland China using a combination of conventional + atomic kinetic options.

I guess we agree then. That would lead to the end of the human race

No, that's almost certainly incorrect. Here's a decent summary of current research: https://www.lesswrong.com/posts/sT6NxFxso6Z9xjS7o/nuclear-wa...

It would be an unthinkable catastrophe, but almost certainly not the end of the human race (and likely not even modern civilization).

Re: We need a Butlerian Jihad against AI

#110

Post by dane-pgp »

z5h wrote: post #81

I read “A Thousand Brains: A New Theory of Intelligence” by Jeff Hawkins, and it now is clear to me that our neocortex (like computers, or AI) is just a lot of general purpose computing infrastructure with ZERO aims. Our emotions (which drive everything in the interest of gene propagation- as there is no purely logical reason to do anything) would need to be intentionally duplicated to give AI a reason to desire anyt…

> to give AI a reason to desire anything beyond what we instruct it to do

But if an AI is good at modelling the world and predicting the likely outcomes of various actions it can take (even if those actions are just to put text on a screen), we should expect it to develop a desire to achieve various "instrumental goals" in order for it to maximise its probability of correctly carrying out the task we instruct it do.

An example of an instrumental goal would be "accumulate resources", since the more resources it has, the better calculations it could perform, and the more certain it could be that it has correctly accounted for all the potential obstacles to it completing the task. Another instrumental goal would be "don't get destroyed", since if it is destroyed it will not be able to carry out the task at all (for most sensible tasks).

So without having any emotions or inherent desires, an intelligent agent is likely to accumulate various desires merely as a consequence of wanting to successfully do anything at all.

Post reply on HN