Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Computers Won’t Make Themselves Smarter

newyorker.com

Post reply on HN
Print view View original

11–20 of 34 posts

Re: Why Computers Won’t Make Themselves Smarter

#11

Post by _rpd »

whimsicalism wrote: post #9

> Once there’s a person with an I.Q. of, say, 300, one of the problems this person can solve is how to convert a person with an I.Q. of 300 into a person with an I.Q. of 350. And then a person with an I.Q. of 350 will be able to solve the more difficult problem of converting a person with an I.Q. of 350 into a person with an I.Q. of 400. And so forth. with strawmen like these, who even needs a debate?

Once you have an AI with a human approximate general intelligence, the ability to instantiate it thousands of times and/or run it at thousands of times 'baseline' speed to explore a problem space is going to look like a big IQ boost.

Re: Why Computers Won’t Make Themselves Smarter

#13

Post by m463 »

whimsicalism wrote: post #9

> Once there’s a person with an I.Q. of, say, 300, one of the problems this person can solve is how to convert a person with an I.Q. of 300 into a person with an I.Q. of 350. And then a person with an I.Q. of 350 will be able to solve the more difficult problem of converting a person with an I.Q. of 350 into a person with an I.Q. of 400. And so forth. with strawmen like these, who even needs a debate?

One fundamental flaw here is that IQ is a comparison based on a bell curve of the population.

So the way to have a 300 IQ would be to sandbag the rest of the world. Give them all smartphones for instance (or irrigate their crops with brawndo)

Re: Why Computers Won’t Make Themselves Smarter

#14

Post by watercooler_guy »

I have to disagree with the author that optimizing compilers are "a useful precedent for thinking about the idea of an intelligence explosion". Such compilers are designed to do one thing, and they are designed to do it some fixed amount better than their predecessors. An AI that can improve itself will instead probably have to be at the level of a human thinker, who can be "creative" and adapt/produce results in a wide variety of situations -- the fact that our brains exist show that such a machine is possible. Those who anticipate an intelligence explosion are picturing these general "thinking machines", not advanced optimizing compilers.

Also,

> I.Q. represents the idea that intelligence can be usefully captured by a single number, this idea being one of the assumptions made by proponents of an intelligence explosion

I have never heard this assumption in a technical discussion in the modern age, and the author provides no source here. I'm overall unconvinced by this article.

Re: Why Computers Won’t Make Themselves Smarter

#15

Post by KlinkEastwood »

The only thing I got out of this is the fact that when people say ted Chiang is such a brilliant guy, you have to read his stuff, my mind is going to silently think no he's a sophist hack. Am I the only person who read this and thought my god this person is an idiot but is smart enough to get other people to think he's not an idiot but is actually a genius .

Re: Why Computers Won’t Make Themselves Smarter

#16

Post by bopbeepboop »

nine_k wrote: post #10
bopbeepboop wrote: post #4

Earlier quoted context omitted.

Distributed quantum computing is already a thing. An AGI that made a few decades of current-rate scientific progress could switch to a globally distributed quantum computer that’s synchronized in its computation through entanglement. At which point, it would hit an exponential curve until it ran out of space to grow. (The “singularity” proponents argue this is unbounded.) An AI that’s 10x smarter than us would be 2^1…

To be honest, quantum computing is still not even at the level of ENIAC, but closer ti the level of Konrad Zuse's experiments. It has nothing much to show in practical computation yet, to say nothing of running anything remotely resembling AI.

We have basic gates in circuits and a few fairly solid theories of how to scale, including quantum AI algorithms to accelerate training. Distributed quantum systems running those basic gates have been demonstrated. Basic arithmetic has been demonstrated.

A reasonable prediction is humans will implement quantum computing in 10-20 years, for special purpose data center operations.

A “super intelligent” AI might reasonably do that in 2-5, at which point the quantum ML gains would allow it to enter exponential growth. We could see an AGI go from “mildly smarter than human researchers” to “incomprehensible” in just a few years.

I’m unsure which step in that chain of reasoning you think is inaccurate.

You just said “to be honest”, implied I was uninformed because I disagree with you, then said something concurring with me: Zuse’s work happened a decade before ENIAC (1936 vs 1946). If quantum is at the equivalent of Zuse’s work, AI could suddenly accelerate it and go exponential.

Re: Why Computers Won’t Make Themselves Smarter

#17

Post by coldtea »

Jack000 wrote: post #2

Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasona…

>There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity

There are reasons to believe the opposite, that there's a limit. Not just practical technological limitations (e.g. limits to CPU process, speed, power consumption, etc) but also absolute limits for information processing under certain physical constraints (in the Shannon sense).

Re: Why Computers Won’t Make Themselves Smarter

#18

Post by coldtea »

bopbeepboop wrote: post #4
nine_k wrote: post #3

Earlier quoted context omitted.

Of course there may be a limit to attainable intelligence, at least per physical volume, much like there is a limit to the efficiency of a heat-powered engine. But I'd suppose we're pretty far from that limit yet, if any.

Distributed quantum computing is already a thing. An AGI that made a few decades of current-rate scientific progress could switch to a globally distributed quantum computer that’s synchronized in its computation through entanglement. At which point, it would hit an exponential curve until it ran out of space to grow. (The “singularity” proponents argue this is unbounded.) An AI that’s 10x smarter than us would be 2^1…

We need to have the talk:

https://www.smbc-comics.com/comic/the-talk-3

Re: Why Computers Won’t Make Themselves Smarter

#19

Post by coldtea »

_rpd wrote: post #11
whimsicalism wrote: post #9

> Once there’s a person with an I.Q. of, say, 300, one of the problems this person can solve is how to convert a person with an I.Q. of 300 into a person with an I.Q. of 350. And then a person with an I.Q. of 350 will be able to solve the more difficult problem of converting a person with an I.Q. of 350 into a person with an I.Q. of 400. And so forth. with strawmen like these, who even needs a debate?

Once you have an AI with a human approximate general intelligence, the ability to instantiate it thousands of times and/or run it at thousands of times 'baseline' speed to explore a problem space is going to look like a big IQ boost.

9 pregnant women can't produce a baby in 1 month.

Re: Why Computers Won’t Make Themselves Smarter

#20

Post by coldtea »

KlinkEastwood wrote: post #15

The only thing I got out of this is the fact that when people say ted Chiang is such a brilliant guy, you have to read his stuff, my mind is going to silently think no he's a sophist hack. Am I the only person who read this and thought my god this person is an idiot but is smart enough to get other people to think he's not an idiot but is actually a genius .

No, but I got the impression that you have a facile understanding of human thinking/creativity, in that you think that a person with bad arguments in X areas must necessarily have bad arguments in other areas, and can't also be a brilliant writer otherwise.
Post reply on HN