Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why traditional reinforcement learning will probably not yield AGI [pdf]

philpapers.org

Post reply on HN
Print view View original

51–60 of 123 posts

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#51

Post by PeterisP »

sebastos wrote: post #45
lacker wrote: post #11

TLDR: reinforcement learning cannot handle AGI, because reinforcement learning rewards must be finite, but a true artificial intelligence could reason about infinite numbers. I think this is complete nonsense. Humans don't receive infinite rewards, either, but we still think about infinite numbers. We typically think about infinite numbers in terms of finite representations, like finite proofs about their properties.…

Is that an accurate TL;DR? I'm pretty sure no... It's not saying you need rewards that are infinite in value per se. It's saying that you might have more situations that need to be differentiated from each other than there are real numbers. For instance, you might need to have one set of rewards that map to the real numbers, and then another set of rewards that also map to the real numbers when compared to themselves…

It's assserting that there's a need to differentiate more situations from each other than there are real numbers, and basing its conclusions on the assertion, but it is not providing a convincing basis why that assertion/assumption/hypothesis is true.

The author demonstrates that a system without such a capability would not be able to solve a certain set of problems. However, going from that claim to a claim that this capability is needed for human level AGI is a non-sequitur - there is no evidence that such a capability is needed for human-level intelligence, and there's no evidence (at least not mentioned in the paper) that humans have the exact capability described.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#52

Post by bra-ket »

xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

do your claims apply in complex-valued RL context?

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#53

Post by [deleted] »

GregarianChild wrote: post #27
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

[deleted]

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#54

Post by bra-ket »

cmrx64 wrote: post #46
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

do you know about the dopamine reward error hypothesis? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6721851/ is it so wrong? what does cognitive psychology have to say about how these neurons work? this is a lot more recent than the 40s and behaviorism.

dopamine rewards operate on a different time scale vs. that required by these error correction models. I don't remember the exact paper, will need to look it up, but it was orders of magnitude difference in response times.

Edit: for authoritative reference on biologically-plausible learning see anything by Edmund Rolls [1]. He explicitly stated in his recent book [2] that something like back-propagation, or similar error correction mechanisms have no supporting evidence in experimental data collected so far

[1] https://www.oxcns.org/profile.html

[2] https://www.amazon.com/Cerebral-Cortex-Principles-Edmund-Rol...

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#55

Post by gwern »

xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

"Assuming AGI agents are Turing computable, no individual AGI can possibly comprehend codes for all computable ordinals, because the set of codes of computable ordinals is badly non-computably-enumerable."

I was going to criticize this paper as crankery in the vein of Penrose, but first I thought I'd just compute all possible ordinals in my brain to make sure I'm a general intelligence.

brb.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#56

Post by AlphaSite »

renjimen wrote: post #48
qyv wrote: post #36

Earlier quoted context omitted.

> But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured. This is true in the context of the universe as a whole, not by the organism itself.

Except no organism is born a blank slate. Parent is correct in that our prior was massively expensive to construct

I think the point huh is being mad is that individual people (or models) dot learn that way. It’s not like models training models, all the way down.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#57

Post by sn41 »

xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

I think you are right in constructing situations where real numbers are inadequate. It is also right that you do not claim that hyperreals or surreals suffice, you are merely pointing out that they may help you to do better than the reals.

But I have often wondered - why are people hung up on linear ordering? Why not non-total partial orders?

https://en.wikipedia.org/wiki/Ordinal_optimization

Is this insistence on linear orders because of simplistic modeling on the part of cognitive science and AI people, or is there some problem with general partial orders?

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#58

Post by [deleted] »

bra-ket wrote: post #33
anotherProd wrote: post #23

Earlier quoted context omitted.

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm. Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" i…

My point was that RL/DL is being used like some kind of massive hammer to hit all the nails. Cognition requires different, specialized, energy-efficient tools. > consciousness All talk about this is premature and "pre-science", before we figure out more basic, fundamental things like object storage and recall from memory, object recognition from sensory input, concept representation and formation, the exact mechanism…

[deleted]

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#60

Post by benchaney »

joe_the_user wrote: post #39
benchaney wrote: post #32

Earlier quoted context omitted.

Your cat was already capable cognition before you started training it. GP is talking about generating a cognition where it did not previously exist.

Why couldn't a mechanism (say A Neural Turing or whatever) that you train be "cognition capable" when you start and then be trained to actual behavior after that?

You would need something that is "cognition capable" first and that has not been invented yet.
Post reply on HN