A Reply to François Chollet on Intelligence Explosion • Hacker News Board

Re: A Reply to François Chollet on Intelligence Explosion

#61

Post by naasking » Fri, Dec 08, 2017, 12:44 AM UTC

aetherson wrote: post #60

naasking wrote: post #53

Earlier quoted context omitted.

> But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia. We are. Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the timeline from the first computer program to play chess, to beating ch…

Or it just seems like that to you because these are the changes that you've viscerally experienced instead of just reading about. Someone born in 1900, looking back at their life in 1975, would be like "When I was born, heavier-than-air flight was impossible. Now people routinely fly across oceans at 600 mph, you can travel faster than the speed of sound for admittedly a lot of money, and we've gone to the motherfuck…

Like the other poster, these are also examples of super-linear progress, so I'm not sure what you think this proves.

For my point, the super-linear progress in information tech is all that's needed to argue in favour of the intelligence explosion.

Re: A Reply to François Chollet on Intelligence Explosion

#62

Post by colordrops » Fri, Dec 08, 2017, 12:50 AM UTC

naasking wrote: post #55

colordrops wrote: post #22

Earlier quoted context omitted.

That makes an assumption that there aren't limits on intelligence regimes and that you can just recursively improve intelligence without much friction. That's a very big assumption that has no basis in evidence.

> That makes an assumption that there aren't limits on intelligence regimes and that you can just recursively improve intelligence without much friction. No it doesn't. AI just needs to get smarter than humans for it to be dangerous to us. The costs C and C' the parent comment described can include an upper bound on the cost of the self-improvement needed to achieve each respective goal.

I don't disagree with you seem to have missed the hypothetical situation I was responding to:

> It can’t, so it spends some time recursively improving itself

Re: A Reply to François Chollet on Intelligence Explosion

#63

Post by aetherson » Fri, Dec 08, 2017, 12:52 AM UTC

naasking wrote: post #61

aetherson wrote: post #60

Earlier quoted context omitted.

Or it just seems like that to you because these are the changes that you've viscerally experienced instead of just reading about. Someone born in 1900, looking back at their life in 1975, would be like "When I was born, heavier-than-air flight was impossible. Now people routinely fly across oceans at 600 mph, you can travel faster than the speed of sound for admittedly a lot of money, and we've gone to the motherfuck…

Like the other poster, these are also examples of super-linear progress, so I'm not sure what you think this proves. For my point, the super-linear progress in information tech is all that's needed to argue in favour of the intelligence explosion.

You seem to think that a short period of fast technology growth in some areas corresponds to the claim that technology as a whole is improving more-than-linearly.

Compare the y values of the functions y = x vs. y = sqrt(x) over the x values in the interval [0..1]. Or the slopes of the lines.

It's been 42 years since 1975. If technological advance was faster from 1900-1975 than it has been from 1976-2017, or "only as fast," then that's important to understand, and probably more relevant to our immediate future than whether technological growth from 8000 BC to 1900 AD was either by some standard very impressive or slower than growth from 1900-2000.

Re: A Reply to François Chollet on Intelligence Explosion

#64

Post by naasking » Fri, Dec 08, 2017, 12:52 AM UTC

colordrops wrote: post #62

naasking wrote: post #55

Earlier quoted context omitted.

> That makes an assumption that there aren't limits on intelligence regimes and that you can just recursively improve intelligence without much friction. No it doesn't. AI just needs to get smarter than humans for it to be dangerous to us. The costs C and C' the parent comment described can include an upper bound on the cost of the self-improvement needed to achieve each respective goal.

I don't disagree with you seem to have missed the hypothetical situation I was responding to: > It can’t, so it spends some time recursively improving itself

I was responding to your response to that. Recursive improvement doesn't need to be unbounded, it just needs to supercede us.

Re: A Reply to François Chollet on Intelligence Explosion

#65

Post by goatlover » Fri, Dec 08, 2017, 12:56 AM UTC

naasking wrote: post #59

evgen wrote: post #58

Earlier quoted context omitted.

> Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the time frame from from the invention of controlled heavier-than-air flight to landing a man on the moon; within a human lifetime and before you were born. Look at the time frame from the point when the vast majority of the human population lived their entire li…

Those are all examples of super-linear progress, so I'm not sure what point you think you've made. > Things that may seem trivial or even primitive to you were far more important and world-changing inventions Which has zero bearing on the point I was making, which is that super-linear progress in information technology is all around us. Information tech is all that matters to the question of general AI. Like I said,…

That's only true if the information technology is leading up to AGI, and not say, augmenting human intelligence instead. One could argue that the intelligence explosion has been happening for centuries, but it's human intelligence not machine, that is being amplified.

Re: A Reply to François Chollet on Intelligence Explosion

#66

Post by goatlover » Fri, Dec 08, 2017, 12:59 AM UTC

mannykannot wrote: post #7

aetherson wrote: post #3

Something to note about the "we're on an exponential (or at least more-than-linear) increase in technology" arguments: They generally need to reach back quite a long time to sound convincing. Yudkowsky makes two arguments about how fast technology or society is evolving: in one he chooses 500 years ago, 1517. In another, he talks about "the last 10,000 years." In contrast, Chollet compares 1900-1950 with 1950-2000. I…

This should be seen in the context of the article Yudkowsky is replying to. Collet's position is that runaway AGI is impossible, and in support of that, he claims that it is impossible for AI technology to grow super-linearly. In his refutation, Yudkowski has merely to show that it is not beyond the bounds of possibility. Of course, insofar as he is also putting forward the view that the risk is significant, he is al…

So the real question is about plausibility, not possibility?

It's possible that predatory aliens will show up on our doorstep sometime in the next few decades. But it's not very likely, and not something we need to prepare for.

Re: A Reply to François Chollet on Intelligence Explosion

#67

Post by renox » Fri, Dec 08, 2017, 1:03 AM UTC

evgen wrote: post #58

naasking wrote: post #53

Earlier quoted context omitted.

> But if we're on a more-than-linear increase, then we should be seeing more and more technological growth in very recent timeframes, not needing to reach back centuries or millennia. We are. Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the timeline from the first computer program to play chess, to beating ch…

> Look at the time frame from the invention of the transistor to everyone doing nearly everything online. It's less than the average human lifetime. Look at the time frame from from the invention of controlled heavier-than-air flight to landing a man on the moon; within a human lifetime and before you were born. Look at the time frame from the point when the vast majority of the human population lived their entire li…

> Look at the time frame from the point when information could travel no faster than the speed of a good horse

Incorrect example: semaphore relays existed already, but yes transoceanic communications happened very fast indeed.

Re: A Reply to François Chollet on Intelligence Explosion

#68

Post by goatlover » Fri, Dec 08, 2017, 1:04 AM UTC

GuiA wrote: post #5

pasquinelli wrote: post #4

Earlier quoted context omitted.

by what means will a cancer research bot take over the world? it doesn't matter how smart you are if you don't have the necessary means to do something. i think it's a fantasy, something that people who imagine themselves to be very intelligent have latched onto-- the idea that their best quality is the best quality.

Well, the general AI doomsday fundamentalist argument proceeds as follows: you tell an AI to cure cancer. It can’t, so it spends some time recursively improving itself, then it finds out that the cost of curing cancer is C, but the cost of killing all humans (and therefore indirectly curing cancer) is C’, where C’ If you’re a smarty pants you tell the AI to cure cancer AND not kill all humans. But because the AI is s…

If the AI is smart enough to manipulate the world to the point of killing us off or putting us all into eternal cryostasis, then it should be smart enough to know that's not what we intended.

Re: A Reply to François Chollet on Intelligence Explosion

#69

Post by goatlover » Fri, Dec 08, 2017, 1:07 AM UTC

Chickenality wrote: post #32

bobthechef wrote: post #31

These kinds of discussions aren't very rigorous and reveal a basic philosophical illiteracy and philistinism at work. There's quite a bit of question begging going on. The elephant in the room is that the prevailing materialistic/naturalistic (MN) understanding of the world is completely impotent where intentionality, qualia, consciousness, etc, are concerned. Philosophers like Thomas Nagel talk about it; the incorri…

I haven't seen anyone argue that intentionality, qualia, or consciousness would necessarily be either a precondition or a result of developing AGI. In fact, thought experiments like the "paperclip maximizer" are often brought up to argue that a machine could be very alien in its internal experience or lack thereof, but still pose an existential threat.

If an AI can turn the world into paperclips, it can certainly understand that we wouldn't want that. Paperclipping everything is a much harder task.

Re: A Reply to François Chollet on Intelligence Explosion

#70

Post by naasking » Fri, Dec 08, 2017, 1:12 AM UTC

aetherson wrote: post #63

naasking wrote: post #61

Earlier quoted context omitted.

Like the other poster, these are also examples of super-linear progress, so I'm not sure what you think this proves. For my point, the super-linear progress in information tech is all that's needed to argue in favour of the intelligence explosion.

You seem to think that a short period of fast technology growth in some areas corresponds to the claim that technology as a whole is improving more-than-linearly. Compare the y values of the functions y = x vs. y = sqrt(x) over the x values in the interval [0..1]. Or the slopes of the lines. It's been 42 years since 1975. If technological advance was faster from 1900-1975 than it has been from 1976-2017, or "only as…

Firstly, growth rates of non-information tech largely isn't relevant to AI. That said, overall growth in knowledge is at least directly proportional to population growth. Even pessimistically considering humans as dumb, an exponential population growth means an exponential discovery rate just from sheer trial and error. So overall progress is undeniably exponential, even if it's a low exponent.

Secondly, we know definitively that information density has been growing exponentially given Moore's law. The much decried end of Moore's law is for a particular incarnation of information tech, but there's still plenty of room to grow in other directions.

Even with our current tech base, we can continue to scale exponentially in horizontal directions with more parallelism (see the rise of core counts, GPU and distributed computing). We're nowhere near the end of that scaling in that direction, let alone longer term innovations like optical and quantum computing.

So really, what possible reasons do we really have for thinking that exponential growth will not continue well past human intelligence? Note, I didn't say infinitely, just well past our intelligence.