Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why traditional reinforcement learning will probably not yield AGI [pdf]

philpapers.org

Post reply on HN
Print view View original

61–70 of 123 posts

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#61

Post by mijoharas »

gnulinux wrote: post #31
LatteLazy wrote: post #22

People love talking about intelligence but I'm yet to see a measurable definition. The best anyone can manage is "you know it when you see it" (like porn). Am I missing something or is this all just a dark road to an unknown destination (possibly just a dead end?)

The value of definability is moot at best imho. "Mathematics" is also not easy to define, there are volumes philosophers of mathematics discussing this, some even say "mathematics is what mathematicians find interesting" (similar to your porn example) but this doesn't stop us from studying mathematics. Same goes for science e.g.. People like Popper or Kuhn spent a lot of mental cycles arguing what is and what is not…

> Same goes for science e.g.. People like Popper or Kuhn spent a lot of mental cycles arguing what is and what is not science, yet people still do science every day without reading them.

I'm quite fond of a terrible mathematical joke which this reminded me of.

"I'm worried about my nephew, I was trying to teach him to add numbers, but he can't even pronounce zermelo fraenkel set theory, how's he ever going to learn it?!"

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#62

Post by bra-ket »

GregarianChild wrote: post #27
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

>Where is .. AI

it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.

I mean you can't just throw things at the wall and hope they stick, but it's literally the state of the art, if you follow ML conferences and their world-beating toy benchmarks results, with a lot of pseudo-rigorous handwaving for theory.

The reason physics has been so successful is that their theory closely followed empirical data and constraints imposed by nature.

I think the only hope to achieve common sense in AI is to align it with hard constraints living organisms have, using those constraints as a guide.

A few terms I mentioned are coming from that POV, if you dig a bit deeper they all have direct physical manifestation in natural learning systems.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#63

Post by xvilka »

anotherProd wrote: post #23
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm. Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" i…

I think it will require at least a few more centuries to build AGI.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#64

Post by Animats »

TTPrograms wrote: post #28
xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

I think you are hiding a key assumption - that an AGI must be capable of performing optimally on a task with Archimedean measure. This is a very strong assumption - performing eps-close to optimal may certainly fall in the definition of AGI, and this might be achieved by a non-Archimedean approximation. Defining AGI as performing optimally on any set of tasks is problematic from a computational theory perspective in…

That an AGI must be capable of performing optimally on a task with Archimedean measure.

If that's correct, this may be just one of those many problems where the optimal solution is far harder than a near-optimal solution. Examples include linear programming and the traveling salesman problem, where a true optimum is NP-hard to find, but you can get very close with far less work.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#65

Post by xamuel »

gwern wrote: post #55
xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

"Assuming AGI agents are Turing computable, no individual AGI can possibly comprehend codes for all computable ordinals, because the set of codes of computable ordinals is badly non-computably-enumerable." I was going to criticize this paper as crankery in the vein of Penrose, but first I thought I'd just compute all possible ordinals in my brain to make sure I'm a general intelligence. brb.

Hi, thanks for looking at my paper. If you're interested in the relation between Lucas-Penrose stuff and enumeratability of ordinal codes, you might like I.J. Good (1969), "Godel's theorem is a red herring" (2 pages). Can you elaborate on what it is about my paper that strikes you as crankery? I'm a fan of yours so it would be much appreciated.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#66

Post by the8472 »

AlphaSite wrote: post #56
renjimen wrote: post #48

Earlier quoted context omitted.

Except no organism is born a blank slate. Parent is correct in that our prior was massively expensive to construct

I think the point huh is being mad is that individual people (or models) dot learn that way. It’s not like models training models, all the way down.

Individual people are not trained from scratch. ML models often have to be (modulo fine-tuning) since the field is still young.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#67

Post by thomasfromcdnjs »

anotherProd wrote: post #23
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm. Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" i…

Stumbled upon this the other day, seems interesting ¯\_(ツ)_/¯

https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind

"Penrose argues that human consciousness is non-algorithmic, and thus is not capable of being modeled by a conventional Turing machine, which includes a digital computer. Penrose hypothesizes that quantum mechanics plays an essential role in the understanding of human consciousness. The collapse of the quantum wavefunction is seen as playing an important role in brain function."

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#68

Post by xamuel »

sn41 wrote: post #57
xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

I think you are right in constructing situations where real numbers are inadequate. It is also right that you do not claim that hyperreals or surreals suffice, you are merely pointing out that they may help you to do better than the reals. But I have often wondered - why are people hung up on linear ordering? Why not non-total partial orders? https://en.wikipedia.org/wiki/Ordinal_optimization Is this insistence on li…

One could certainly contemplate versions of RL with non-linear orderings. I guess the reason people care about linear ordering is because you want the agent to at least understand "this outcome is better than that outcome". How would we hope for a good nonlinear-RL agent to behave in an environment with 2 buttons, one of which always gives reward X, and the other of which always gives reward Y, where X and Y are incomparable?

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#69

Post by xamuel »

TTPrograms wrote: post #28
xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

I think you are hiding a key assumption - that an AGI must be capable of performing optimally on a task with Archimedean measure. This is a very strong assumption - performing eps-close to optimal may certainly fall in the definition of AGI, and this might be achieved by a non-Archimedean approximation. Defining AGI as performing optimally on any set of tasks is problematic from a computational theory perspective in…

Hi, thanks for looking at my paper. I do not assume that an AGI must be capable of performing optimally on all tasks in general--indeed, that's quite impossible. When measuring the performance of RL agents, one must come up with some way of aggregating performance across many environments, but that's beside the point of this paper. The point of this paper is that if you're forced to use real numbers as rewards, you can't even communicate all environments to the agent without misleading the agent. Whether the agent could perform well or poorly in the environments once communicated, is beside the point.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#70

Post by dkersten »

renjimen wrote: post #48
qyv wrote: post #36

Earlier quoted context omitted.

> But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured. This is true in the context of the universe as a whole, not by the organism itself.

Except no organism is born a blank slate. Parent is correct in that our prior was massively expensive to construct

So we can expect our ANN’s to yield AGI in a few million or billion years? That doesn’t sound like a good place to put our current efforts then.
Post reply on HN