Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why traditional reinforcement learning will probably not yield AGI [pdf]

philpapers.org

Post reply on HN
Print view View original

101–110 of 123 posts

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#101

Post by dkersten »

mannykannot wrote: post #98
YeGoblynQueenne wrote: post #94

Earlier quoted context omitted.

Why wouldn't it follow? Human intelligence evolved in the real world with all its vast information content. Deep learning systems are only trained on a few terrabytes of data of a single type (images, text, sound etc). Even if they can be trained faster than the rate at which animals evolved, their training data is so poor, compared to the "data" that "trained" animal intelligence that we'll be lucky if we can arrive…

You elided the "necessarily". One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

Not grounds for stopping work[1], but perhaps grounds to explore other avenues[2] to see if something else might yield faster results.

I’m no expert, but my personal opinion is that AGI will probably be some hybrid approach that uses some reinforcement learning mixed with other techniques. At the very least, I think an AGI will need to exist in an interactive environment rather than just trained on preset datasets. Prior context or not, a child doesn’t learn by being shown a lot of images, it learns by being able to poke at the world to see what happens. I think an AGI will likely require some aspect of that (and apply reinforcement learning that way).

But like I said, I’m no expert and that’s just my layperson opinion.

[1] if the goal is AGI, if it’s not then of course there’s no reason to stop

[2] some people are doing just that, of course

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#102

Post by YeGoblynQueenne »

mannykannot wrote: post #98
YeGoblynQueenne wrote: post #94

Earlier quoted context omitted.

Why wouldn't it follow? Human intelligence evolved in the real world with all its vast information content. Deep learning systems are only trained on a few terrabytes of data of a single type (images, text, sound etc). Even if they can be trained faster than the rate at which animals evolved, their training data is so poor, compared to the "data" that "trained" animal intelligence that we'll be lucky if we can arrive…

You elided the "necessarily". One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

>> You elided the "necessarily".

Well, if it follows, then it follows necessarily. But maybe that's just a deformation professionelle? I spend a lot of time working with automated theorem proving where there's no ifs and buts about conclusions following from premises.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#103

Post by [deleted] »

jula432vdf wrote: post #100

So cockroaches have most probably not a big neural network, but they are able to do lots of stuff, including reproduction. I think this kind of machine intelligence is already at reach of currents models, or we are close to be able to make an "e-cockroach". I think we'll learn lots of stuff just seconds after having put in the world that kind of limited artificial intelligence. And then the models, through sensors, w…

[deleted]

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#104

Post by lostmsu »

sebastos wrote: post #37
lostmsu wrote: post #15

Earlier quoted context omitted.

I don't see how representation of floats by natural numbers that we all use on daily basis is misleading. In fact, your entire comment is just an ordered sequence of natural numbers, and it does not seem to be very misleading (though it tried to trick). Besides, the article is purposed to be a rigorous mathematical proof of current representations in RL being unsuitable for AGI, but I haven't seen a Definition for "m…

?? Current reinforcement learning algorithms don't score an action by producing a line of text that is read by a human. It assigns a value by writing a floating point number into a register on the computer somewhere. If it were otherwise, and there was some kind of interpretation unit that was needed to compare two rewards and decide which one was bigger, then hey, you're not using the reals for your rewards anymore,…

In that interpretation the article basically says "you can't precisely optimize function F in domain X with by optimizing function F' in domain Y, when cardinality of Y Well, obviously! That is basically the definition of cardinality!

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#105

Post by visarga »

xamuel wrote: post #7
nx2059 wrote: post #3

Earlier quoted context omitted.

The article was pretty over my head, but does your argument hold to the various augmented neural net systems such as Neural Turing Machines, or Differentiable Neural Computers?

The argument isn't so much about the type of agent (which I think is what Neural Turing Machines etc. are about), it's about the type of environment. In traditional Reinforcement Learning, environments give real-number-valued rewards (or even rational-number-valued rewards which is even more constrained). Presumably this was a decision that was made with hardly a second thought because real numbers are most familiar…

There are multi-reward agents and multi-task agents but all rewards get added into a final scalar value. And gradient based methods need to have this one scalar value to derive gradients from.

Since you mentioned higher dimensional representations for rewards, I want to remind that the sub-fields of Inverse RL and Model-based RL are concerned with reward representation and prediction by neural nets.

Also, it doesn't seem like a good idea to try to disprove an entire field with a purely theoretical (a-priori) argument. There should be at least some consideration given to the state of the art in the field.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#106

Post by gwern »

xamuel wrote: post #65
gwern wrote: post #55

Earlier quoted context omitted.

"Assuming AGI agents are Turing computable, no individual AGI can possibly comprehend codes for all computable ordinals, because the set of codes of computable ordinals is badly non-computably-enumerable." I was going to criticize this paper as crankery in the vein of Penrose, but first I thought I'd just compute all possible ordinals in my brain to make sure I'm a general intelligence. brb.

Hi, thanks for looking at my paper. If you're interested in the relation between Lucas-Penrose stuff and enumeratability of ordinal codes, you might like I.J. Good (1969), "Godel's theorem is a red herring" (2 pages). Can you elaborate on what it is about my paper that strikes you as crankery? I'm a fan of yours so it would be much appreciated.

Sadly, I was only able to count through a vanishingly small subset of the reals in a finite time, and therefore am not a general intelligence, and so it would be foolish of me, a machine made of a handful of atoms, to try to criticize this paper. It sure would be nice if I could appreciate music, but you've proven that's impossible, so it is what it is.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#107

Post by mannykannot »

YeGoblynQueenne wrote: post #102
mannykannot wrote: post #98

Earlier quoted context omitted.

You elided the "necessarily". One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

>> You elided the "necessarily". Well, if it follows, then it follows necessarily. But maybe that's just a deformation professionelle? I spend a lot of time working with automated theorem proving where there's no ifs and buts about conclusions following from premises.

If I am not mistaken, it does not necessarily follow unless it turns out to be a sound argument in every possible world.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#108

Post by mannykannot »

dkersten wrote: post #101
mannykannot wrote: post #98

Earlier quoted context omitted.

You elided the "necessarily". One can rationally argue either way over the speculative proposition that reinforcement learning will yield AI in less than a few million years, but that it took evolution half a billion years is hardly conclusive, and certainly not grounds for stopping work.

Not grounds for stopping work[1], but perhaps grounds to explore other avenues[2] to see if something else might yield faster results. I’m no expert, but my personal opinion is that AGI will probably be some hybrid approach that uses some reinforcement learning mixed with other techniques. At the very least, I think an AGI will need to exist in an interactive environment rather than just trained on preset datasets. P…

Fair enough, though I do not think the evidence from evolution moves the needle much with respect to the timeline. For one thing, evolution was not dedicated to the achievement of intelligence.

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#109

Post by YeGoblynQueenne »

mannykannot wrote: post #107
YeGoblynQueenne wrote: post #102

Earlier quoted context omitted.

>> You elided the "necessarily". Well, if it follows, then it follows necessarily. But maybe that's just a deformation professionelle? I spend a lot of time working with automated theorem proving where there's no ifs and buts about conclusions following from premises.

If I am not mistaken, it does not necessarily follow unless it turns out to be a sound argument in every possible world.

Ah, so you are making a formal argument? In that case you should stick to formal language. And probably publish it in a different venue :)

Re: Why traditional reinforcement learning will probably not yield AGI [pdf]

#110

Post by mannykannot »

YeGoblynQueenne wrote: post #109
mannykannot wrote: post #107

Earlier quoted context omitted.

If I am not mistaken, it does not necessarily follow unless it turns out to be a sound argument in every possible world.

Ah, so you are making a formal argument? In that case you should stick to formal language. And probably publish it in a different venue :)

No, I am simply responding to your rather formal point, in kind. Unless you are aguing for it being an established fact that the time evolution took to produce intelligent life rules out any form of reinforcement learning producing AI in any remotely reasonable period of time, then that original point of yours does not seem to be going anywhere.

In your work on theorem proving, am I right in guessing that there are no 'ifs' or 'buts' because the truth of premises is not an issue? In the "evolution argument", the premises/lemmas are not just that evolution took a long time, but also something along the lines of significant speedup not being possible.

You might notice that in another comment, I suggested that we might still be in the AI Cambrian. I'm not being inconsistent, as no-one knows for sure one way or the other.

Post reply on HN