211–220 of 312 posts
Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.
Earlier quoted context omitted.
Why are forgetting and sleep relevant? If someone invented a pill that gave you a perfect memory and removed the need to sleep, would you stop being generally intelligent if you took it?
Because they are strongly associated with all known examples of generalized intelligence. Why wouldn’t they be relevant?
Correlation != Causation. While they very likely might be relevant, I've not seen anything to conclusively prove that it is. The ability to forget is important to humans because we are emotional beings, but I don't think that necessarily is a requirement for generalized intelligence. "sleep" (as in what happens during sleep, not the act specifically itself) on the other hand is very likely important, but again, not proven.
Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.
They did avoid one common pitfall at least. They are (intentionally?) vague about which number systems the rewards can come from, apparently leaving it open whether the rewards need be real-valued or whether they can be, say, hyperreals, surreals, computable ordinals, etc. This avoids a trap I've written about elsewhere [1]: traditionally, RL rewards are limited to be real-valued (usually rational-valued). I argue that RL with real-valued rewards is NOT enough to reach AGI, because the real numbers have a constrained structure making them not flexible enough to express certain goals which an AGI should nevertheless have no problem comprehending (whether or not the AGI can actually solve them---that's a different question). In other words: if real-valued RL is enough for AGI, but real-valued RL is strictly less expressive than more general RL, then what is more general RL good enough for? "Artificial Better-Than-General Intelligence"?
Note, however, that almost all [2] practical RL agent technology (certainly any based on neural nets or backprop) very fundamentally assumes real-valued rewards. So if it is true that "RL is enough" but also that "real-valued RL is not enough", then the bad news is all that progress on real-valued RL is not guaranteed to help us reach AGI.
[1] "The Archimedean trap: Why traditional reinforcement learning will probably not yield AGI", JAGI 2020, https://philpapers.org/archive/ALETAT-12.pdf
[2] A notable exception is preference-based RL
Earlier quoted context omitted.
Really? I created two GI's and it wasn't very hard and was actually quite fun. Training them is a bit of a pain though. I'm willing to bet that based on total calories consumed they are amazingly efficient compared to their hypothetical AGI counterparts.
Yes but can they: - live forever - grow their own mental capabilities exponentially over that unlimited lifespan - turn themselves into universe-eating von Neumann probes
- live forever
- grow exponentially forever
- "eat the universe" (I know, the last point was sci-fi gibberish)
In fact, humans are already pretty good at reproducing themselves and have managed to travel to space, and have exhibited finite periods of exponential knowledge growth combined with periods of collapse, as nothing grows exponentially forever.
Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.
Does AGI imply human-level intelligence, or would the intelligence of a housefly qualify?
So yeah, even getting that level of intelligence would be a huge win. However, most people mean close to human level intelligence when they mean AGI even if it's one narrow specialization.
Earlier quoted context omitted.
I don't know who Patricia Churchland is, but they said that the paper was "very carefully and insightfully worked out." After having read the paper, I am very disappointed in the output. Nothing concrete was shown, just hypothesis and reads more like philosophy. That being said, I would say that the paper is carefully worked out and does provide insight if you haven't thought about RL before.
If Patricia Churchland doesn't have a problem with the paper despite it being philosophical, then that's probably because she is a philosopher. An eliminative materialist, to be precise. Personally, from reading the abstract, I disagree with the hypothesis. There's a trick where anything (even say, a database lookup) looks like optimization as long as you contrive the objective function just right, but that's kind of…
Earlier quoted context omitted.
That is if you believe biological general intelligence is the end goal of evolution, which I believe is highly unlikely. Intelligence is simply a special side-product of evolution, there is nothing general about general intelligence. Many organisms can thrive without it. There is also a non-negligible chance that all organisms would die out before reaching intelligence. We are fortunate to live in a world that produc…
> That is if you believe biological general intelligence is the end goal of evolution, which I believe is highly unlikely. I would agree, but might add that evolution doesn't have 'goals'. Is that the point you were trying to make?
Convergent evolution exists for at least some adaptations though, like the eye. It’s not unreasonable to think that there may be some sort of equivalent convergence which creates a high general intelligence adaptation given enough time, at least for social creatures.
I think it’s pretty much impossible to know whether intelligence is a convergent adaptation without some kind of perfect simulation of evolution over billions of years. You’d have to tweak starting conditions and see if you kept getting smart creatures.
Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.
Does AGI imply human-level intelligence, or would the intelligence of a housefly qualify?
Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.
I think the title of the paper makes more sense if you consider that ten years ago, someone could have written a paper in a similar spirit with a different take on "what is enough". Back then, it would probably have been titled: "Backpropagation of errors is enough". The last ten years have shown that backpropagation -- while a crucial component -- is not enough. Personally, I would not be shocked to find out in the…
Earlier quoted context omitted.
I think the title of the paper makes more sense if you consider that ten years ago, someone could have written a paper in a similar spirit with a different take on "what is enough". Back then, it would probably have been titled: "Backpropagation of errors is enough". The last ten years have shown that backpropagation -- while a crucial component -- is not enough. Personally, I would not be shocked to find out in the…
Why are forgetting and sleep relevant? If someone invented a pill that gave you a perfect memory and removed the need to sleep, would you stop being generally intelligent if you took it?
There's a sweet spot between knowing enough and knowing little enough so that you get the right answer and get it quickly enough.