Why traditional reinforcement learning will probably not yield AGI [pdf]

Agreed. In terms of base assumptions, running AGI on computers also puts us at a disadvantage due to multiple layers of abstraction. That is to say, AGI is what organic brains are as opposed to what we're asking computers to do. It's like a complexity difference between building a machine to throw a ball and building one to accurately simulate the physics of throwing a soft rubber ball through turbulent air onto grass. A dragonfly uses just sixteen neurons to take visual input from thousands of ommatidia and use it to track prey in 3D space. How many transistors would we have to use to accomplish the same task? Now scale that up to 86 billion neurons in a human brain. That's the scale of the problem we're looking at after we figure out how to program it.

As someone who's a software engineer (not data scientist) but is interested in consciousness and by extension AGI and dabbled in some ML algorithms, I find it surprising how often I see the sentiments of AGI being possible or impossible using some sort of algorithm.

Obviously I could be missing some great breadth and depth of research (there's definitely a lot I don't know) but from what I've read "we have no idea" is a pretty accurate description with how far we've come when it comes to consciousness, and I would imagine even less for the newer field of AI/AGI (consciousness has been around for a while P: and our theories have mostly sidestepped this real world phenomenon).

> "The idea of using "rewards" as a learning mechanism and a path to actual cognition is just wrong, full stop."

This to me is a huge red flag (mostly of ego/hubris). I think if we rephrased the goal to not talk about "AGI" and maybe around quantitative things like the things you've listed ("computational efficiency", likelihood of being stuck in local minimimas, etc) then I'd happily concede that we should be looking at "X" and not "Y" but unless I've missed something, again likely, when we're talking about AGI, we're talking about consciousness (epiphenomenon that come about through physical/deterministic interactions). A quick way to gut check myself here is twisting what you state is not a good place to start "ML/AI field... gets stuck in local minima" and ask myself is it possible that local minima (which we consider "bad" for current/traditional tasks) could be necessary for consciousness ? I think the widely accepted answer to this is currently "We don't know".

If I think that achieving AGI is going to be similar to what the algorithms and architecture we currently use (where the likelihood of being stuck in a local minima is something we can look at) then sure, your opinions stand. But that is just a guess and unless I'm mistaken AGI hasn't been achieved because we don't know how to do it.

This isn't to say that we should have 100% of the data before making strong judgements like this about a subject. It's just that the subject of "consciousness" is a big one (I'd say THE big one) so making such strong statements about something we know we don't know much about is interesting. <- this is where I get flashbacks to SE world where a missing piece of data can really throw you off or leads to wrong assumptions and when I think about consciousness we know we don't know a lot.

> "This 'Skinnerism' has been discredited in cognitive psychology decades ago and makes absolutely no biological sense whatsoever for the simple reason that any organism trying to adapt in this way will be eaten by predators before minimizing its "error function" sufficiently."

> "Living learning organisms have limited resources (energy and time), and they cut the search space drastically through shortcuts and heuristics and hardcoded biases instead of doing some kind of brute force optimization."

But those heuristics and hardcoded biases were developed through brute force optimization over the course of billions of years, a massive amount of energy input and many organisms being devoured.

Reinforcement learning is Turing complete [1], so if AI is possible at all, then it can be realised through RL.

   cognitive psychology

You are overselling the insights of this discipline. Has cognitive psychology solved its replication problems? Where is the world-beating AI that is based on "concept spaces", "sparse representations", "small-world networks" and "learning and memory" neuroscience?

[1] https://arxiv.org/abs/1505.00521

I've long thought that to reproduce animal-like intelligence artificially, you need to simulate an environment and stimuli, some unequivocally positive and some unequivocally negative.

Of course AGI does not need to be "animal-like intelligence".