Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence cannot be contained: Lessons from Computability Theory

arxiv.org

Post reply on HN
Print view View original

221–227 of 227 posts

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#221

Post by klmadfejno »

qayxc wrote: post #212
klmadfejno wrote: post #189

Earlier quoted context omitted.

This is a bad example. The intelligence needed to carry out your described outcomes is more or less AGI already, and that is advanced enough that describing it as a dumb optimization machine is likely naive. Further, if you want to optimize stamp collection, you don't tell a model what it can't do, you tell it what it can do. It can make trades on stampcollectors.com, for +/- 20% of asking price, max 30 trades per da…

> The intelligence needed to carry out your described outcomes is more or less AGI already Yes! And that's exactly the point at which AI safety comes into play. Nobody is afraid a of a dumb NN that generates texts or images given an input or a classifier. That's not what the paper is concerned with, either. > Further, if you want to optimize stamp collection, you don't tell a model what it can't do, you tell it what…

You're making the assumption that AGI level intelligence would resemble RL bots of today. It's a classic sci fi trope, but likely not one that makes any sense imo.

AGI might require physical instantiation, but that doesn't mean that physical bots are indicative of AGI outcomes. Most of our physical bots are just classically written programs with a bit of ML to help with classification of its environment anyway.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#223

Post by semi-extrinsic »

plutonorm wrote: post #188
semi-extrinsic wrote: post #185

Earlier quoted context omitted.

Difference is, people were able to propose detailed mechanisms for realistic flying machines long before we actually achieved powered flight - that was mainly a matter of increasing the power to weight ratio of the propulsion system. For AGI, do you really think there are detailed proposals out there today that can achieve AGI but are only missing the computational power? Actually the existence of human brains with t…

> For AGI, do you really think there are detailed proposals out there today that can achieve AGI but are only missing the computational power? Yes, I really do. It's neural networks, nothing more. All that is required is more power. Despite its lower power the brain is much more computationally powerful than even the largest supercomputers. Although it is not necessary, imo, to have a much more efficient computing su…

> It is fairly obvious to many now, after the continued scaling of the GPT-x series models, that genuine intelligence is an emergent property of the kind of systems we are building.

I respectfully disagree. GPT-x series models are performing interpolation on an unfathomably massive corpus. It is not hard to find cases where it directly reproduces entire paragraphs from existing text. When given a prompt on a topic for which it finds multiple existing texts with similar degree of matching, such as different articles reporting on the same topic, it is able to blend the content of those articles smoothly.

I mean, GPT-3 is around 6 trillion bits of compressed data. The entire human brain has 0.1 trillion neurons, and it obviously has a capacity far beyond GPT-3 - even in the extreme case if we assume all the neurons in the human brain are used for generating English written text.

In my view GPT-x is very, very far from any kind of general intelligence.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#224

Post by Born_Again »

michaelmrose wrote: post #220
Born_Again wrote: post #206

Earlier quoted context omitted.

> What does "inseparable" mean? That the sensation occurs at the same time that the neurons fire? That may be true, but it doesn't make them equivalent. Can a sensation exist without neurons firing? The root of our conversation is the question if a sensation purely exists in the physical world. If it does, then it is possible to measure it. If it doesn't, then that breaks our scientific understanding of the world and…

These discussions are normally expositions of how the other party misunderstands reality and or terminology with a dash of if i don't understand it but can vaguely describe it then it must be inexplicable.

I agree. I am also not cut out to be a philosopher.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#225

Post by plutonorm »

semi-extrinsic wrote: post #223
plutonorm wrote: post #188

Earlier quoted context omitted.

> For AGI, do you really think there are detailed proposals out there today that can achieve AGI but are only missing the computational power? Yes, I really do. It's neural networks, nothing more. All that is required is more power. Despite its lower power the brain is much more computationally powerful than even the largest supercomputers. Although it is not necessary, imo, to have a much more efficient computing su…

> It is fairly obvious to many now, after the continued scaling of the GPT-x series models, that genuine intelligence is an emergent property of the kind of systems we are building. I respectfully disagree. GPT-x series models are performing interpolation on an unfathomably massive corpus. It is not hard to find cases where it directly reproduces entire paragraphs from existing text. When given a prompt on a topic fo…

> I respectfully disagree

Cool :)

> The entire human brain has 0.1 trillion neurons

You want to be thinking about synapses. There's about 7000 synapses per neuron, so that's 7000 * 0.1 = 700 Trillion synapses. So thats *100 times larger than GPT-3. Also consider that a neuron does a fair amount of processing within the neuron, there is some very recent research on this, each neuron is a akin to a mini neural network. So I would not be surprised if the human brain is 10,000 times more powerful than GPT-3.

> It is not hard to find cases where it directly reproduces entire paragraphs from existing text. When given a prompt on a topic for which it finds multiple existing texts with similar degree of matching, such as different articles reporting on the same topic, it is able to blend the content of those articles smoothly.

This may be true, but it does not prove your hypothesis that all GPT-x models are simply "performing interpolation". Also the ability to perform recall better than a human may be to do with the way that we perform global optimisation over the network, rather than the local decentralised way that the brain presumably works. Point is accurate memorisation does not preclude general intelligence. Spend some time with the models, sit down for a few hours and investigate what they know and do not know, really look, see beyond what you expect to see. You may be surprised.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#226

Post by geocar »

benji-york wrote: post #198
geocar wrote: post #154

Earlier quoted context omitted.

> for the same reason humans largely don't think about flies. What's to think about? They're just an annoyance, to be swatted aside. We can choose to think of flies; to imagine they are as people, with their own (short) lives, wants and wishes and dreams; that there is fly art and fly culture equal to our own. We can so imbue those flies with our animism and ethics, and treat them the way we should (ethically) treat…

If you haven't read it, I think you would really enjoy the book Blindsight ( https://archive.org/details/PeterWattsBlindsight/page/n3/mod... ).

I just finished reading it a few minutes ago; you were right, I did really enjoy it. Thank you.

Re: Superintelligence cannot be contained: Lessons from Computability Theory

#227

Post by benji-york »

geocar wrote: post #226
benji-york wrote: post #198

Earlier quoted context omitted.

If you haven't read it, I think you would really enjoy the book Blindsight ( https://archive.org/details/PeterWattsBlindsight/page/n3/mod... ).

I just finished reading it a few minutes ago; you were right, I did really enjoy it. Thank you.

I'm happy to hear it!
Post reply on HN