Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind says reinforcement learning is ‘enough’ to reach general AI

venturebeat.com

Post reply on HN
Print view View original

141–150 of 312 posts

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#141

Post by tarsinge »

rollcat wrote: post #121
tsimionescu wrote: post #116

Earlier quoted context omitted.

The fact that RL in the extremely vague sense used in the article is enough for AGI is uncontroversial for anyone who believes intelligence and consciousness are physical processes. However, this "result" is trivial. It is obviously equivalent to the claim that intelligence arose naturally in the biological world without influence from God.

> It is obviously equivalent to the claim that intelligence arose naturally in the biological world without influence from God. Where did God's intelligence come from?

Nowhere, that's kind of the definition of God (for a Christian at least): it always was, and is the ultimate origin. It is different than a direct influence in the world.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#143

Post by [deleted] »

chrisco255 wrote: post #83
A-Train wrote: post #76

Earlier quoted context omitted.

The algorithms to train, initialize the networks, new architectures are far more important than the hardware advances. If people knew how to train NNs 50 years ago we would live in a different world.

We did. they just didn't have the same computation abilities back then.

[deleted]

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#144

Post by tsimionescu »

Frost1x wrote: post #140
tsimionescu wrote: post #116

Earlier quoted context omitted.

The fact that RL in the extremely vague sense used in the article is enough for AGI is uncontroversial for anyone who believes intelligence and consciousness are physical processes. However, this "result" is trivial. It is obviously equivalent to the claim that intelligence arose naturally in the biological world without influence from God.

The problem with this, specifically the assumption that RL gives an equivilance to natural selection and evolution, is that RL typically assumes a computational environment it interacts in while natural selection and evolution assumes the physical world as the environment. The important difference here is that in order for RL to translate to solving real world problems, you need to faithfully and computafionally simu…

You are absolutely right about RL in practice. In fact, if we were to look at actual RL algorithms, I believe the paper's claims fall flat in many other ways. This is actually my criticism of it: its arguments are only convincing when RL is defined only as the extremely general notion of an agent seeking to maximize some reward function by interacting with an environment. This is so comically general that the only alternative I can think of is to posit a transcendental god.

Once we get into the details, their claims stop being iron clad. Even worse, some of their claims become actually hard or impossible to accept if applied to actual RL algorithms we have today. You give one good example with the difficulty of modeling the world. The implicit claim they make that this would be realizable in reasonable time (say, less than a billion years) is also not well supported. The idea that humans or mammals learn their social behaviors through RL rather than a good deal of reasoning from evolutionarily-trained first principles pretty clearly fails in the face of the poverty of the stimulus argument[0].

Overall, the claims in the paper tend to switch between obvious (if taken to talk about the general idea of maximizing reward) to almost certainly wrong (if taken to talk about known RL algorithms, reasonable time frames, and specific examples of what is supposed to be learned).

[0] the poverty of the stimulus argument may be controversial in linguistics where it was first formulated. Still, if applied to mammal or insect socialization, the extremely low time frames in which individuals of a species start exhibiting typical behaviors basically proves in my opinion that they are instincts, trained at the population level through evolution, not individual learning through RL. The extreme similarity of behavior between individuals of the same species, VS the variety of behaviors between different species, also suggests an important component of species-level rather than individual level learning.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#145

Post by tinco »

croon wrote: post #93
tinco wrote: post #80

Earlier quoted context omitted.

It is entirely not necessary for an AGI to be able to drive a car. Frankly, after seeing AlphaZero and AlphaFold I'm surprised they didn't declare AGI right there and then. People assume that when AGI happens, computers can suddenly outsmart humans in every way and solve every problem imaginable. The reality is just that it could in theory given enough time and resources. It is like quantum computing. In theory it ca…

> It is entirely not necessary for an AGI to be able to drive a car. "Artificial general intelligence (AGI) is the hypothetical[1] ability of an intelligent agent to understand or learn any intellectual task that a human being can." What definition are you using?

The same, that they understand and can learn how to drive a car doesn't mean they would actually be able to do it in the real world.

You can read a book on how to hit a ball with a baseball bat, you can even practice and get good at it, but that still doesn't mean you would actually be able to hit a ball thrown by a professional pitcher.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#146

Post by croon »

tinco wrote: post #145
croon wrote: post #93

Earlier quoted context omitted.

> It is entirely not necessary for an AGI to be able to drive a car. "Artificial general intelligence (AGI) is the hypothetical[1] ability of an intelligent agent to understand or learn any intellectual task that a human being can." What definition are you using?

The same, that they understand and can learn how to drive a car doesn't mean they would actually be able to do it in the real world. You can read a book on how to hit a ball with a baseball bat, you can even practice and get good at it, but that still doesn't mean you would actually be able to hit a ball thrown by a professional pitcher.

The interface to the car is a solved problem.

> You can read a book on how to hit a ball with a baseball bat, you can even practice and get good at it, but that still doesn't mean you would actually be able to hit a ball thrown by a professional pitcher.

If I hade incredibly fast reflexes and actuators I could.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#147

Post by tinco »

qsort wrote: post #100
tinco wrote: post #80

Earlier quoted context omitted.

It is entirely not necessary for an AGI to be able to drive a car. Frankly, after seeing AlphaZero and AlphaFold I'm surprised they didn't declare AGI right there and then. People assume that when AGI happens, computers can suddenly outsmart humans in every way and solve every problem imaginable. The reality is just that it could in theory given enough time and resources. It is like quantum computing. In theory it ca…

> it could in theory given enough time and resources. In theory given enough time and resources, anyone can defeat any grandmaster in Chess: just compute the extended tree form of the game and run the minimax algorithm. The "given enough time and resources" clause makes everything that follows meaningless, unless a reasonable algorithm is presented. > It is like quantum computing. It is absolutely not like quantum co…

> In theory given enough time and resources, anyone can defeat any grandmaster in Chess: just compute the extended tree form of the game and run the minimax algorithm.

Yes, that's why we're considered to be generally intelligent. It is exactly the point, and not at all meaningless. Right now there's no machine that can come up with the idea to run an extended tree form of the game and minimax the algorithm. If there was such a machine, then that machine would be considered AGI.

> It is absolutely not like quantum computing.

I meant in the sense that just that it has actually been achieved, it doesn't mean it's as powerful as we have described in the theory. In theory you can use Shor's algorithm to break encryption, in practice the devices we have today have trouble with 2 digit numbers.

The same principle goes for AGI. If someone releases an AGI system today, it doesn't mean that tomorrow we'll see a Boston Dynamics robot hop on a bicycle to his day job as a Disney movie art director. The world would most likely not change at all, at least not for a while, many people would not recognise the significance and many people might not even recognise the fact that it is in fact AGI.

> As far as AGI goes, we have absolutely no idea. There's lively debate on whether anything we have done even counts as significant advancement towards AGI.

You might think that, and that says something about what side of the debate you're on. We're commenting here on the thread of an article about DeepMind asseting that reinforcement learning is enough to reach general AI. If that's true (and I think it is), then we've probably reached general AI already.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#148

Post by tarvaina »

rich_sasha wrote: post #101
d110af5ccf wrote: post #78

Earlier quoted context omitted.

Deep Mind is cutting edge ML in general, right? Doesn't Google actively apply the lessons learned all over the place? YouTube content recommendation stands out to me in particular. Translation and automated closed captioning are also obviously ML based. I'd guess that most of the really interesting stuff would be behind the scenes and not immediately visible to end users though.

I’m stressing the business part. YouTube is a loss-making business year after year. Deep Mind gloss doesn’t seem to change that. If indeed it even is Deep Mind making those improvements, Google has lots of other ML groups, such as Google Brain, and these are more directly focused on Google products. There’s no denying their academic success, or game playing etc, but as far as I can see, the data centre cooling bit is…

YouTube made $6B revenue in Q1. [1] While they don’t release profit numbers, it would be pretty surprising if they were negative.

Did you mean to write DeepMind instead? If so, I don’t disagree.

[1] https://www.cnbc.com/2021/04/27/youtube-could-soon-equal-net...

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#149

Post by tinco »

croon wrote: post #146
tinco wrote: post #145

Earlier quoted context omitted.

The same, that they understand and can learn how to drive a car doesn't mean they would actually be able to do it in the real world. You can read a book on how to hit a ball with a baseball bat, you can even practice and get good at it, but that still doesn't mean you would actually be able to hit a ball thrown by a professional pitcher.

The interface to the car is a solved problem. > You can read a book on how to hit a ball with a baseball bat, you can even practice and get good at it, but that still doesn't mean you would actually be able to hit a ball thrown by a professional pitcher. If I hade incredibly fast reflexes and actuators I could.

Similarly, DeepMind's software might be able to drive a car, would it have a similar neuron count, connectivity, perception systems and training you received.

Or maybe it couldn't, because the software is not as efficient as the organisation of your brain is. Or because there's hardcoded routines evolved in your brain that it lacks.

What I'm saying is that just that because an AGI can't drive a car, it doesn't mean it's an AGI. For the same reason there's loads of people out there that are generally intelligent that can't drive cars for all sorts of physical reasons.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#150

Post by robomartin »

Yeah. No. No way.

My son and I were discussing the state of AI/robotics yesterday, as we walked a beautiful trail in the Sequoia National Forest.

What prompted the discussion was a simple question:

What would it take to build a robot capable of navigating these trails as we do?

This would be a robot able to do this in a manner indistinguishable from, say, a ten year old human.

No GPS, maps, compass, pre-mapping, lidar, ultrasonic sensors, etc. Just vision, hearing and touch/force sensing at the “skin” and articulations.

What do you know?

Two things: You are located at the start of the correct trail and there’s a waterfall at the end.

Our conclusion was equally short and simple: Today, it is hopelessly impossible to match what a ten year old kid could do on that trail.

Maybe in ten years. Maybe.

AI today can’t do what a ten year old human, or the young bears we so along the trail, can do instantly and without thinking: Understand.

We just don’t know how to approach and encode understanding yet.

Post reply on HN