Why Computers Won’t Make Themselves Smarter
newyorker.com
Why Computers Won’t Make Themselves Smarter
1–10 of 34 posts
Re: Why Computers Won’t Make Themselves Smarter
#2Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense.
There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasonable to believe that 1 - agi is possible 2 - said agi can participate in human society to improve its own design
Re: Why Computers Won’t Make Themselves Smarter
#3Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasona…
Of course there may be a limit to attainable intelligence, at least per physical volume, much like there is a limit to the efficiency of a heat-powered engine. But I'd suppose we're pretty far from that limit yet, if any.
Re: Why Computers Won’t Make Themselves Smarter
#4Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasona…
Of course there may be a limit to attainable intelligence, at least per physical volume, much like there is a limit to the efficiency of a heat-powered engine. But I'd suppose we're pretty far from that limit yet, if any.
Distributed quantum computing is already a thing.
An AGI that made a few decades of current-rate scientific progress could switch to a globally distributed quantum computer that’s synchronized in its computation through entanglement.
At which point, it would hit an exponential curve until it ran out of space to grow. (The “singularity” proponents argue this is unbounded.)
An AI that’s 10x smarter than us would be 2^10 smarter than us in a couple years — based on current technology trends. Moreover, it will have overcome the cooperation barrier with a distributed consciousnes, like the original idea for the Borg.
The ability to free itself from a single location or easy disconnection from its assets would be a major step in AI independence.
Re: Why Computers Won’t Make Themselves Smarter
#5Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasona…
> and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense.
The central limit theorem shows that the normal distribution emerges as the limit of the average of independent and identically distributed random variables of any distribution.
Re: Why Computers Won’t Make Themselves Smarter
#6Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasona…
> and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. The central limit theorem shows that the normal distribution emerges as the limit of the average of independent and identically distributed random variables of any distribution.
And there is no particular reason to believe that capabilities are merely the sum of a bunch of random iid variables! If you look at how DL scales with model size or data or FLOPS, it's a powerlaw (or lognormal), and in countless places in the natural world, things are lognormal rather than normal. Similarly, wealth, scientific productivity, lots of things are lognormal rather than normal. Even for human intelligence, the normality is due more to the fact that we all come off the assembly line as Mark 1 human units, practically identical, with the differences coming from environmental randomness & randomness in genetic mutation assortment.
Re: Why Computers Won’t Make Themselves Smarter
#7The word evolution didn’t appear in this article which is odd considering it’s the mechanism responsible for getting us from a single cell to an individual capable of saying such advancement is unlikely.
Re: Why Computers Won’t Make Themselves Smarter
#8Surprised this was written by Ted Chiang (my all-time favorite science-fiction author). The IQ and compiler arguments seem like pretty weak strawmen, and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. There's no reason to believe that intelligence is a quantity that can be improved ad infinitum ala singularity, but I think it's reasona…
> and the last section assumes that AI will have some kind of normal distribution of IQ just like humans, which doesn't make any sense. The central limit theorem shows that the normal distribution emerges as the limit of the average of independent and identically distributed random variables of any distribution.
Won't AI designers choose optimal values for those variables that have been identified? If so, I don't think the result looks like a normal curve, or at least the resultant 'optimized' normal curves skew 'smarter' relative to the human normal curve.
Re: Why Computers Won’t Make Themselves Smarter
#9> Once there’s a person with an I.Q. of, say, 300, one of the problems this person can solve is how to convert a person with an I.Q. of 300 into a person with an I.Q. of 350. And then a person with an I.Q. of 350 will be able to solve the more difficult problem of converting a person with an I.Q. of 350 into a person with an I.Q. of 400. And so forth.
with strawmen like these, who even needs a debate?
Re: Why Computers Won’t Make Themselves Smarter
#10Earlier quoted context omitted.
Of course there may be a limit to attainable intelligence, at least per physical volume, much like there is a limit to the efficiency of a heat-powered engine. But I'd suppose we're pretty far from that limit yet, if any.
Distributed quantum computing is already a thing. An AGI that made a few decades of current-rate scientific progress could switch to a globally distributed quantum computer that’s synchronized in its computation through entanglement. At which point, it would hit an exponential curve until it ran out of space to grow. (The “singularity” proponents argue this is unbounded.) An AI that’s 10x smarter than us would be 2^1…
To be honest, quantum computing is still not even at the level of ENIAC, but closer ti the level of Konrad Zuse's experiments. It has nothing much to show in practical computation yet, to say nothing of running anything remotely resembling AI.