Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why traditional reinforcement learning will probably not yield AGI [pdf]

philpapers.org

Post reply on HN
Print view View original

41–50 of 123 posts

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#41

Post by odnes »

bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

> The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent.

A lot of research in RL is focused on intrinsic motivation and the question of whether we can bootstrap our own 'rewards' from our ability to predict and control the future according to some self-defined goals/hypotheses.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#42

Post by dumbfoundded »

lacker wrote: post #11

TLDR: reinforcement learning cannot handle AGI, because reinforcement learning rewards must be finite, but a true artificial intelligence could reason about infinite numbers. I think this is complete nonsense. Humans don't receive infinite rewards, either, but we still think about infinite numbers. We typically think about infinite numbers in terms of finite representations, like finite proofs about their properties.…

Humans do have infinite rewards (or at least penalties). Death is a pretty common one for example.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#43

Post by fsuuttg »

GregarianChild wrote: post #27
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

Why does it matter? Turing completeness is a low bar. Both exp(exp(N)) and log(log(N)) algorithms solve the AI problem in a finite number of steps, but one of them is a useless abstraction, while the other one really works.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#44

Post by GregarianChild »

ForHackernews wrote: post #35
GregarianChild wrote: post #27

Earlier quoted context omitted.

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

> Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. This seems like overstating your point. Nobody has been able to rigorously define "AI" yet, so there's no way of saying whether it's possible with a Turing machine architecture. The human brain, at least, doesn't seem that similar to a Turing architecture. Neurons don't carry out anything like discrete op…

While I agree that we don't have an agreed upon definition of AI, the problem is firmly in the "I" part of AI! The "A" part is taken to mean implementable by a computer, i.e. a Turing machine. This is the content of the Church–Turing thesis [1].

[1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#45

Post by sebastos »

lacker wrote: post #11

TLDR: reinforcement learning cannot handle AGI, because reinforcement learning rewards must be finite, but a true artificial intelligence could reason about infinite numbers. I think this is complete nonsense. Humans don't receive infinite rewards, either, but we still think about infinite numbers. We typically think about infinite numbers in terms of finite representations, like finite proofs about their properties.…

Is that an accurate TL;DR? I'm pretty sure no...

It's not saying you need rewards that are infinite in value per se. It's saying that you might have more situations that need to be differentiated from each other than there are real numbers. For instance, you might need to have one set of rewards that map to the real numbers, and then another set of rewards that also map to the real numbers when compared to themselves, but are all considered strictly greater than the first set.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#46

Post by cmrx64 »

bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

do you know about the dopamine reward error hypothesis? https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6721851/ is it so wrong? what does cognitive psychology have to say about how these neurons work? this is a lot more recent than the 40s and behaviorism.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#47

Post by joe_the_user »

ForHackernews wrote: post #35
GregarianChild wrote: post #27

Earlier quoted context omitted.

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

> Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. This seems like overstating your point. Nobody has been able to rigorously define "AI" yet, so there's no way of saying whether it's possible with a Turing machine architecture. The human brain, at least, doesn't seem that similar to a Turing architecture. Neurons don't carry out anything like discrete op…

Maybe it's possible to run AGI on a Turing machine, maybe it's not,

Arguing from ignorance, of course nothing is knowable for certain. However there has been a lot of work on the universality of Turing machines, showing that a Turing machine can simulate any conceivable concept of finite computation and can approximate any conventional physical system.

I think a more useful way to express your intuition is to note that if human-built AGI comes into existence, it might be runnable on a Turing machine but quite possibly not efficiently so.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#48

Post by renjimen »

qyv wrote: post #36
postnihilism wrote: post #25

Earlier quoted context omitted.

> "This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently." > "Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuris…

> But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured. This is true in the context of the universe as a whole, not by the organism itself.

Except no organism is born a blank slate. Parent is correct in that our prior was massively expensive to construct

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#49

Post by YeGoblynQueenne »

GregarianChild wrote: post #27
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

Note that Turing complete means undecidable. An algorithm that can learn any program that can be computed by a Universal Turing Machine must, in the worst case, search an infinite program space. So, even if a Neural Turing Machine can learn arbitrary programs (I haven't read the paper so I can't say) it might need to consume infinite resources before learning any particular progarm.

In short- Turing completeness is no guaranteed path to AGI. Assuming an "AGI program" exists, it is hidden away in an infinity of almost identical, but not quite, programs.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#50

Post by PeterisP »

dumbfoundded wrote: post #42
lacker wrote: post #11

TLDR: reinforcement learning cannot handle AGI, because reinforcement learning rewards must be finite, but a true artificial intelligence could reason about infinite numbers. I think this is complete nonsense. Humans don't receive infinite rewards, either, but we still think about infinite numbers. We typically think about infinite numbers in terms of finite representations, like finite proofs about their properties.…

Humans do have infinite rewards (or at least penalties). Death is a pretty common one for example.

We do not treat death as an infinite penalty in our decisionmaking, as evidenced by various cases of people choosing to die for one reason or another, prioritizing the achievement of some other goal (with a finite reward) over survival.

People can and do use statements like "death must be avoided at all costs" (it's also mentioned in the paper), but that statement simply is hyperbole which is not exactly literally true description of reality.

Post reply on HN