Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Reply to François Chollet on Intelligence Explosion

intelligence.org

Post reply on HN
Print view View original

21–30 of 89 posts

Re: A Reply to François Chollet on Intelligence Explosion

#21

Post by colordrops »

pasquinelli wrote: post #4
skybrian wrote: post #2

The more I read about this, the more I think we should stick to narrow AI. If you don't want your cancer research bot to take over the world, don't give it general reasoning capability. Goal alignment seems incredibly fragile in comparison.

by what means will a cancer research bot take over the world? it doesn't matter how smart you are if you don't have the necessary means to do something. i think it's a fantasy, something that people who imagine themselves to be very intelligent have latched onto-- the idea that their best quality is the best quality.

People run cartels from jail cells. Even if an AGI is confined to a machine with no actuators, all it would need is internet access to affect the world. Even without internet access you can cross the gap, as seen with the Stuxnet malware.

Re: A Reply to François Chollet on Intelligence Explosion

#22

Post by colordrops »

GuiA wrote: post #5
pasquinelli wrote: post #4

Earlier quoted context omitted.

by what means will a cancer research bot take over the world? it doesn't matter how smart you are if you don't have the necessary means to do something. i think it's a fantasy, something that people who imagine themselves to be very intelligent have latched onto-- the idea that their best quality is the best quality.

Well, the general AI doomsday fundamentalist argument proceeds as follows: you tell an AI to cure cancer. It can’t, so it spends some time recursively improving itself, then it finds out that the cost of curing cancer is C, but the cost of killing all humans (and therefore indirectly curing cancer) is C’, where C’ If you’re a smarty pants you tell the AI to cure cancer AND not kill all humans. But because the AI is s…

That makes an assumption that there aren't limits on intelligence regimes and that you can just recursively improve intelligence without much friction. That's a very big assumption that has no basis in evidence.

Re: A Reply to François Chollet on Intelligence Explosion

#23

Post by dekhn »

LolWolf wrote: post #8
aetherson wrote: post #3

Something to note about the "we're on an exponential (or at least more-than-linear) increase in technology" arguments: They generally need to reach back quite a long time to sound convincing. Yudkowsky makes two arguments about how fast technology or society is evolving: in one he chooses 500 years ago, 1517. In another, he talks about "the last 10,000 years." In contrast, Chollet compares 1900-1950 with 1950-2000. I…

Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.

sin(x) = x for very small x, this is a useful tool for several proofs.

Re: A Reply to François Chollet on Intelligence Explosion

#24

Post by cdancette »

dekhn wrote: post #23
LolWolf wrote: post #8

Earlier quoted context omitted.

Just for funsies (but may answer your question): every (differentiable) function is linear at a small enough time scale. This includes e^x or pretty much every function you listed.

sin(x) = x for very small x, this is a useful tool for several proofs.

for all functions, f(x) = f(0) + f'(0) * x for x very close to 0

Re: A Reply to François Chollet on Intelligence Explosion

#25

Post by [deleted] »

nkrisc wrote: post #17
yorwba wrote: post #14

Earlier quoted context omitted.

"The researchers read the printed report of the solution and decide" ... to implement it immediately. It will save so many lives! They only need to manufacture a few specific molecules to assemble them into nanobots and then ... where is that gray goo coming from?!

I guess that's the risk of blindly trusting AI. Of course, the AI did not forcibly destroy us in that scenario. Trust, but verify. If the AI's solution is so complex as to be beyond human understanding, well, that's a different issue.

[deleted]

Re: A Reply to François Chollet on Intelligence Explosion

#26

Post by mundo »

nkrisc wrote: post #9
GuiA wrote: post #5

Earlier quoted context omitted.

Well, the general AI doomsday fundamentalist argument proceeds as follows: you tell an AI to cure cancer. It can’t, so it spends some time recursively improving itself, then it finds out that the cost of curing cancer is C, but the cost of killing all humans (and therefore indirectly curing cancer) is C’, where C’ If you’re a smarty pants you tell the AI to cure cancer AND not kill all humans. But because the AI is s…

I think the question is, and the one I have too, is why would such AI have any ability to do anything beyond output a Cure Cancer solution to a terminal? Why does the AI need to be the one to implement its derived solution? The AI has a good think and comes up with a solution of killing all humans. The researchers read the printed report of the solution and decide against implementing it, tweak parameters, and ask th…

I find "killer robots destroy the world" a lot easier to imagine than "all humans collectively agree to adopt common-sense safeguards on AI research even though they limit potential corporate profits".

Re: A Reply to François Chollet on Intelligence Explosion

#27

Post by Smaug123 »

cdancette wrote: post #24
dekhn wrote: post #23

Earlier quoted context omitted.

sin(x) = x for very small x, this is a useful tool for several proofs.

for all functions, f(x) = f(0) + f'(0) * x for x very close to 0

This is a specific instance of what LolWolf just said in the grandparent.

Re: A Reply to François Chollet on Intelligence Explosion

#29

Post by vbarrielle »

Lots of points can be reduced to the ability to simulate a physical environment very fast, and much faster than the actual occuring of events. But it doesn't look like it's easy to simulate physics at high speeds, let alone faster than what happens in our environment. Therefore we are bound by our environment and our limited ability to simulate it.
Post reply on HN