Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Reply to François Chollet on Intelligence Explosion

intelligence.org

Post reply on HN
Print view View original

11–20 of 89 posts

Re: A Reply to François Chollet on Intelligence Explosion

#11

Post by AndrewKemendo »

skybrian wrote: post #2

The more I read about this, the more I think we should stick to narrow AI. If you don't want your cancer research bot to take over the world, don't give it general reasoning capability. Goal alignment seems incredibly fragile in comparison.

It's silly because it ignores all of the mechanisms required for the doomsday scenario. You need some automated killing machine of some sort and if it hasn't been built, then an AGI would have to compel some non-AGI entity to build it or somehow takeover the construction of such a machine.

Re: A Reply to François Chollet on Intelligence Explosion

#12

Post by aetherson »

LolWolf wrote: post #8
aetherson wrote: post #3

Something to note about the "we're on an exponential (or at least more-than-linear) increase in technology" arguments: They generally need to reach back quite a long time to sound convincing. Yudkowsky makes two arguments about how fast technology or society is evolving: in one he chooses 500 years ago, 1517. In another, he talks about "the last 10,000 years." In contrast, Chollet compares 1900-1950 with 1950-2000. I…

Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.

Only on infinitely small subintervals of x.

Re: A Reply to François Chollet on Intelligence Explosion

#14

Post by yorwba »

nkrisc wrote: post #9
GuiA wrote: post #5

Earlier quoted context omitted.

Well, the general AI doomsday fundamentalist argument proceeds as follows: you tell an AI to cure cancer. It can’t, so it spends some time recursively improving itself, then it finds out that the cost of curing cancer is C, but the cost of killing all humans (and therefore indirectly curing cancer) is C’, where C’ If you’re a smarty pants you tell the AI to cure cancer AND not kill all humans. But because the AI is s…

I think the question is, and the one I have too, is why would such AI have any ability to do anything beyond output a Cure Cancer solution to a terminal? Why does the AI need to be the one to implement its derived solution? The AI has a good think and comes up with a solution of killing all humans. The researchers read the printed report of the solution and decide against implementing it, tweak parameters, and ask th…

"The researchers read the printed report of the solution and decide" ... to implement it immediately. It will save so many lives! They only need to manufacture a few specific molecules to assemble them into nanobots and then ... where is that gray goo coming from?!

Re: A Reply to François Chollet on Intelligence Explosion

#15

Post by LolWolf »

aetherson wrote: post #12
LolWolf wrote: post #8

Earlier quoted context omitted.

Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.

Only on infinitely small subintervals of x.

This approximation has vanishing error from a linear function (as the interval decreases) and (most importantly!) we only have finite noisy samples; so it’s essentially indistinguishable (in a statistical way) for a small enough interval given some variance.

Re: A Reply to François Chollet on Intelligence Explosion

#16

Post by dang »

SandB0x wrote: post #13

Reminder that Eliezer Yudowsky is a crank https://rationalwiki.org/wiki/Eliezer_Yudkowsky

This is a self-refuting comment in terms of its value for HN: even assuming you're 100% right, it's neither civil nor substantive. Please don't post like this.

Re: A Reply to François Chollet on Intelligence Explosion

#17

Post by nkrisc »

yorwba wrote: post #14
nkrisc wrote: post #9

Earlier quoted context omitted.

I think the question is, and the one I have too, is why would such AI have any ability to do anything beyond output a Cure Cancer solution to a terminal? Why does the AI need to be the one to implement its derived solution? The AI has a good think and comes up with a solution of killing all humans. The researchers read the printed report of the solution and decide against implementing it, tweak parameters, and ask th…

"The researchers read the printed report of the solution and decide" ... to implement it immediately. It will save so many lives! They only need to manufacture a few specific molecules to assemble them into nanobots and then ... where is that gray goo coming from?!

I guess that's the risk of blindly trusting AI. Of course, the AI did not forcibly destroy us in that scenario. Trust, but verify.

If the AI's solution is so complex as to be beyond human understanding, well, that's a different issue.

Re: A Reply to François Chollet on Intelligence Explosion

#18

Post by zarkov99 »

pasquinelli wrote: post #4
skybrian wrote: post #2

The more I read about this, the more I think we should stick to narrow AI. If you don't want your cancer research bot to take over the world, don't give it general reasoning capability. Goal alignment seems incredibly fragile in comparison.

by what means will a cancer research bot take over the world? it doesn't matter how smart you are if you don't have the necessary means to do something. i think it's a fantasy, something that people who imagine themselves to be very intelligent have latched onto-- the idea that their best quality is the best quality.

Read "Life 3.0" for a number of simple but very plausible scenarios for how super intelligence could escape containment.

Re: A Reply to François Chollet on Intelligence Explosion

#19

Post by mundo »

pasquinelli wrote: post #4
skybrian wrote: post #2

The more I read about this, the more I think we should stick to narrow AI. If you don't want your cancer research bot to take over the world, don't give it general reasoning capability. Goal alignment seems incredibly fragile in comparison.

by what means will a cancer research bot take over the world? it doesn't matter how smart you are if you don't have the necessary means to do something. i think it's a fantasy, something that people who imagine themselves to be very intelligent have latched onto-- the idea that their best quality is the best quality.

a) If it can't surprise you, it's not really intelligent

b) If it can surprise you, it can do so negatively

That's all you need to demonstrate that the danger exists, that an AI can mis-use the tools you give to it. The simplicity of it makes it pretty irrefutable.

Separate from that, the extent of the danger depends entirely on the details of what the AI does and what it's hooked up. Sure, an AI that can't do anything except output text to a screen isn't very scary. The assumption AI-threat types are making is that we wouldn't be paranoid enough to limit the AIs we work on in that way; we would use them to do things like drive cars or route airline traffic or design our cpus, where "negative surprises" can have disastrous consequences.

Re: A Reply to François Chollet on Intelligence Explosion

#20

Post by sitkack »

The AGIs won't take over the world. Humans and corporations will use narrow AIs to do much worse (better) before that happens. The intelligence explosion that I would like to see centers around widely distributing tools that make formal methods easier to use. Humans so far have a lock on constructive creativity, AI and computation could augment that creativity by effortlessly checking our work. It could make the cognitive work of all humans more rigorous.
Post reply on HN