Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why traditional reinforcement learning will probably not yield AGI [pdf]

philpapers.org

Post reply on HN
Print view View original

91–100 of 123 posts

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#91

Post by IshKebab »

eli_gottlieb wrote: post #81
IshKebab wrote: post #38

Earlier quoted context omitted.

But isn't the sophisticated structure what leads to the real number. If you change the rewards to something more complex surely you still have to pick between actions and at some point you'll have to evaluate which one is "better" and I can't see why you couldn't use real numbers to represent utility. I mean, humans are general intelligences, and you can translate pretty much any human reward into money, which is a r…

>I mean, humans are general intelligences, and you can translate pretty much any human reward into money, which is a real number. A lot of people have written quite a lot of arguments that this is false.

A lot of people have written a lot of arguments about everything. Has anyone actually demonstrated that it isn't true?

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#92

Post by ForHackernews »

GregarianChild wrote: post #44
ForHackernews wrote: post #35

Earlier quoted context omitted.

> Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. This seems like overstating your point. Nobody has been able to rigorously define "AI" yet, so there's no way of saying whether it's possible with a Turing machine architecture. The human brain, at least, doesn't seem that similar to a Turing architecture. Neurons don't carry out anything like discrete op…

While I agree that we don't have an agreed upon definition of AI, the problem is firmly in the "I" part of AI! The "A" part is taken to mean implementable by a computer, i.e. a Turing machine. This is the content of the Church–Turing thesis [1]. [1] https://en.wikipedia.org/wiki/Church%E2%80%93Turing_thesis

> It states that a function on the natural numbers can be calculated by an effective method if and only if it is computable by a Turing machine.

The article at the top of this thread is specifically about properties ("The Generalized Archimedean Property") that real numbers do not possess.

There's also a little bit of slipperiness around the use of "AI" vs. "AGI" - you could easily argue (and people do!) that we've already achieved "AI" for many specialized domains. It's the General bit that seems to be the sticking point, and that this article focuses on.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#93

Post by YeGoblynQueenne »

bra-ket wrote: post #62
GregarianChild wrote: post #27

Earlier quoted context omitted.

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL. cognitive psychology You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience? [1] https://arxiv.org/…

>Where is .. AI it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents. I mean you can't just throw things at the wall and hope they stick, but it's literally the state of the art, if you follow ML conferences and their world-beating toy benchmarks results, with a lot of pseudo-rigorous handwaving for theory. The reason physics has been so successful is that their…

>> it's always 5 years away because mainstream AI researches are stuck with yak shaving their gradient descents.

A small correction: that's deep learning researches, not AI researchers and not all machine learning researchers even. To be charitable, it's not even all deep learning researchers. It's just that the field of deep learning research has been inundated with new entrants who are sufficiently skilled to grok the practicalities but lack understanding of AI scholarship and produce unfortunately shoddy work that does not advance the field (any field, any of the aforementioned ones).

As a personal example, my current PhD studies are in Inductive Logic Programming which is, in short, machine-learning of logic programs (you know, Prolog etc). I would not be able to publish any papers without a theoretical section with actual theoretical results (i.e. theorems and their proofs - and it better be a theorem other than "more parameters beget better accuracy", which is not really a theorem). Reviewers would just reject such a paper without second thought, regardless of how many leaderboards I beat in my empirical results section.

And of course there are all the other fields of AI were work continues - search, classical planning, constraint satisfaction, automated theorem proving, knowledge engineering and so on and so forth.

Bottom line- the shoddy scholarship you flag up does not characterise the field of AI research, as a whole, it only afflicts a majority of modern deep learning research.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#94

Post by YeGoblynQueenne »

mannykannot wrote: post #76
dkersten wrote: post #70

Earlier quoted context omitted.

So we can expect our ANN’s to yield AGI in a few million or billion years? That doesn’t sound like a good place to put our current efforts then.

That does not necessarily follow, as I imagine you well know.

Why wouldn't it follow? Human intelligence evolved in the real world with all its vast information content. Deep learning systems are only trained on a few terrabytes of data of a single type (images, text, sound etc). Even if they can be trained faster than the rate at which animals evolved, their training data is so poor, compared to the "data" that "trained" animal intelligence that we'll be lucky if we can arrive at anything comparable to animal intelligence by deep learning in a billion years.

Or unlucky, as the case may be.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#95

Post by YeGoblynQueenne »

gwern wrote: post #55
xamuel wrote: post #2

Author here. One particularly topical observation (topical because HN has recently featured discussions about ways to merge statistical and symbolic approaches to AI), from Section 4.2.3: certain cutting-edge number systems such as Conway's "surreal numbers" are so sophisticated that they require lots of symbolic logic just to do basic operations with. Of course this makes these number systems hard to work with, but…

"Assuming AGI agents are Turing computable, no individual AGI can possibly comprehend codes for all computable ordinals, because the set of codes of computable ordinals is badly non-computably-enumerable." I was going to criticize this paper as crankery in the vein of Penrose, but first I thought I'd just compute all possible ordinals in my brain to make sure I'm a general intelligence. brb.

None, one, many, all.

Hah. Still got it.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#96

Post by nagaa »

anotherProd wrote: post #23
bra-ket wrote: post #19

I applaud the effort but the problem with RL as a model of learning is in the definition of RL itself. The idea of using "rewards" as a primary learning mechanism and a path to actual cognition is just wrong, full stop. It's a wrong level of abstraction and is too wasteful in energy spent. Looking at it from CogSci perspective it is essentially an offshoot of behaviorism, using a coarse and extremely inefficient mode…

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm. Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" i…

I have learned to play piano and drive a car. IMO both of these took two completely different sets of systems and algorithms in order to accomplish the learning task. Nothing I learned from piano applies to driving and vice versa. The only thing in common is my brain. We want a computer though to apply those algorithms I learned driving and playing piano to golf and have it work. We will then have "AGI". Obviously that fails. Obviously.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#97

Post by thechao »

thomasfromcdnjs wrote: post #67
anotherProd wrote: post #23

Earlier quoted context omitted.

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm. Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" i…

Stumbled upon this the other day, seems interesting ¯\_(ツ)_/¯ https://en.wikipedia.org/wiki/The_Emperor%27s_New_Mind "Penrose argues that human consciousness is non-algorithmic, and thus is not capable of being modeled by a conventional Turing machine, which includes a digital computer. Penrose hypothesizes that quantum mechanics plays an essential role in the understanding of human consciousness. The collapse of the…

Penrose is a superlatively brilliant physicist, but his opinion on AI is the worst sort of woo. It’s little better than reading Chopra. His argument is most salient as an example of attempting to use authority in one field to garner authority in another.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#98

Post by mannykannot »

YeGoblynQueenne wrote: post #94
mannykannot wrote: post #76

Earlier quoted context omitted.

That does not necessarily follow, as I imagine you well know.

Why wouldn't it follow? Human intelligence evolved in the real world with all its vast information content. Deep learning systems are only trained on a few terrabytes of data of a single type (images, text, sound etc). Even if they can be trained faster than the rate at which animals evolved, their training data is so poor, compared to the "data" that "trained" animal intelligence that we'll be lucky if we can arrive…

You elided the "necessarily".

One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#99

Post by PeterisP »

xamuel wrote: post #73
PeterisP wrote: post #20

Earlier quoted context omitted.

Humans don't receive infinite rewards because any "reward system" in our brain is implemented by receiving a finite amount of electrochemical pleasure signals over a finite amount of time. Human-equivalent minds can obviously be implemented on top a framework that does not have inherently infinite or infinitely divisible values, because human minds are implemented on a top of a substrate that uses a finite amount of…

>Humans don't receive infinite rewards because any "reward system" in our brain is implemented by receiving a finite amount of electrochemical pleasure signals over a finite amount of time. This is like saying computers can't represent infinity because they have only finitely many bytes. Suppose the treasury rewarded you a "superdollar", which is a special object that allows you to create any number of dollars that y…

The worth of a superdollar is equal to as much dollars as you can spend per second times the seconds you can expect to live. Both numbers are large but finite, the first is bounded by the value of global economy (no matter how many dollars you create, the total purchasing power of these dollars can't grow beyond that), the second is bounded if not to a few hundred years then by the time until the heat death of the universe.

In a similar manner, the total amount of reward that you might ever get is capped by the amount of pleasure your brain can perceive at any given moment (which is finite) times your lifespan (which is finite).

You might reason and hypothesise about agents perceiving infinite rewards (as we are doing now), but this has nothing to do with the reality of homo sapiens rewards system(s), or, in fact, the rewards system of any agent existing in our physical reality, which is effectively bounded both in time and space.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#100

Post by jula432vdf »

So cockroaches have most probably not a big neural network, but they are able to do lots of stuff, including reproduction.

I think this kind of machine intelligence is already at reach of currents models, or we are close to be able to make an "e-cockroach".

I think we'll learn lots of stuff just seconds after having put in the world that kind of limited artificial intelligence.

And then the models, through sensors, will have the entire world at reach to begin auto-improvement tasks.

And this would solve the issue "we're just training the models with very limited data, how could they evolve faster than millions of years?"

I think the FAANG, the big players, realized this limitation a LOT time ago (10 years maybe).

And they are already trying some things to solve the limited data issue, giving their models all the information they can extract from cellphones, the current iteration of "massive network of sensors to train BIG - multiple, almost hidden from the public - models.

If I have to bet, I'd bet the whole information is being stored to be "replayed" when whole new more advanced models emerge eventually.

Post reply on HN