Live data from Hacker News

DeepMind says reinforcement learning is ‘enough’ to reach general AI

venturebeat.com

111–120 of 312 posts

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#111

Earlier quoted context omitted.

Is Deep Mind a “booming business”? They are achieving great things academically, but their business successes are either kept secret or mostly absent. All I know about is the Google data centre cooling scheduling, probably a big saving for Google but hardly an achievement that in its own professes their business success.

Deep Mind is cutting edge ML in general, right? Doesn't Google actively apply the lessons learned all over the place? YouTube content recommendation stands out to me in particular. Translation and automated closed captioning are also obviously ML based. I'd guess that most of the really interesting stuff would be behind the scenes and not immediately visible to end users though.

No not really. Deep Mind is almost all cutting edge agent-oriented reinforcement learning, hence the nature of the claim they're making. The impact on Google's business from AI has come almost exclusively from other kinds of ML, or that's at least how it appears from the outside. E.g. replacing Google Translate with neural translation doesn't seem to involve RL and certainly doesn't involve agents playing video games.

Deep Mind is best understood as the following bet: if we can train an AI that can learn from "its environment" and do the sort of things a human would do in that situation, then we have achieved AGI and from that ... business ... will follow. Hence their focus on video games as a training environment.

This sounds intuitive but is actually a very agent-centric viewpoint and most AI doesn't resemble this type of thing at all. Most AI deployed so far doesn't have anything resembling an environment, doesn't have any kind of nexus of agency and doesn't need to actively make decisions that then feed back to its own learning, only make probabilistic predictions. And in fact you often don't want an ML model to train on the outcomes of its own decisions.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#112

For those of you ‘generalised’ AI sceptics like me, I can highly recommend reading Søren Brier’s book on Cybersemiotics (“why information is not enough”). It’s a comprehensive reader into the physicalistic, reductionist field of AI and all of its shortcomings. General AI implies the ability to abduct (not just deduction and induction), which I highly doubt will ever be possible.

"never" (I know you didn't say that, but it's implied), is a pretty strong claim. Like with Searle's Chinese Room, I find these kinds of impossibility arguments fairly weak and full of semantic problems. Whenever people argue "consciousness", or "general AI", or "meaning", it's always an exercise in moving the goal posts.

When I read abstracts like this: "Cybersemiotics constructs a non-reductionist framework in order to integrate third person knowledge from the exact sciences and the life sciences with first person knowledge described as the qualities of feeling in humanities and second person intersubjective knowledge of the partly linguistic communicative interactions, on which the social and cultural aspects of reality are based. The modern view of the universe as made through evolution in irreversible time, forces us to view man as a product of evolution and therefore an observer from inside the universe. This changes the way we conceptualize the problem and the role of consciousness in nature and culture. The theory of evolution forces us to conceive the natural and social sciences as well as the humanities together in one theoretical framework of unrestricted or absolute naturalism, where consciousness as well as culture is part of nature. But the theories of the phenomenological life world and the hermeneutics of the meaning of communication seem to defy classical scientific explanations. "

My confidence isn't increased that this book has anything interesting to say. It sounds like the kind of post-modernist verbiage coming out of a lot of humanities departments. I mean seriously, why anyone in a STEM field actually write a sentence like "modern view of the universe as made through evolution in irreversible time, forces us to view man as a product of evolution and therefore an observer from inside the universe." Like what is "irreversible time" adding there? It's kind of redundant, and the awe-inspiring implication: man is a product of evolution, and therefore an observer in the universe? Earth shattering. And all phenomena, from fundamental life sciences to culture, are part of nature? No one's ever considered that before.

My guess is, if you asked a STEM major to write this abstract, they could sum it up with 70% less words. They'd also drop implicitly obvious claims that no one disputes.

I'm betting that this is book is just a less philosophically rigorous version of Searle's argument, given the implication that emergent phenomena "seem to defy classical scientific explanations".

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#113

Everything old is new again. As far as I can tell, they're not actually proposing how to achieve this. I can't access the article without a host institution it seems (is there another link?), so I only have the article to go by. RL has been the basis for all robots engaging with the world, and that engagement with the physical world modeled using RL has been promised to make robots that can act like a 2 year old for…

The paper is creative commons licensed no signup necessary to download

https://www.sciencedirect.com/science/article/pii/S000437022...

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#114
post #3

Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.

That is if you believe biological general intelligence is the end goal of evolution, which I believe is highly unlikely.

Intelligence is simply a special side-product of evolution, there is nothing general about general intelligence. Many organisms can thrive without it.

There is also a non-negligible chance that all organisms would die out before reaching intelligence. We are fortunate to live in a world that produced us.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#115
post #5

Wait isn’t any Turing complete programming language sufficient to eventually reach general AI

Technically that is a conjecture known as the Church-Turing thesis, which states that any computable function can be computed by a Turing machine. Here "computable function" uses the informal definition 'any function that a human can compute', not the formal definition of the recursive functions (which are proven to be solvable by a Turing machine).

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#116
post #18

Some Bozo who has heard all this many times before is suspicious of claims from places like Deep Mind who have a financial incentive to make them (keep funding) where there aren't working machines to back that claim up. Some Bozo has no credentials, no reputation, no track record of publications and barely supports the claim they're making with anything much. Some Bozo has no financial incentives or otherwise to opin…

The fact that RL in the extremely vague sense used in the article is enough for AGI is uncontroversial for anyone who believes intelligence and consciousness are physical processes.

However, this "result" is trivial. It is obviously equivalent to the claim that intelligence arose naturally in the biological world without influence from God.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#117
post #18

Some Bozo who has heard all this many times before is suspicious of claims from places like Deep Mind who have a financial incentive to make them (keep funding) where there aren't working machines to back that claim up. Some Bozo has no credentials, no reputation, no track record of publications and barely supports the claim they're making with anything much. Some Bozo has no financial incentives or otherwise to opin…

Unfortunately i'm rooting for the Bozo, the current AI Revolution won't lead us anywhere and will ebb down eventually.

What do you mean by 'won't lead us anywhere'?

It might or might not give us AGI. But it is already leading us to lots of places. Eg speech recognition even on my phone works way better than what I had twenty years ago on a Desktop.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#118

Basically, any problem with a solution fits into RL: reward of 1 if you are AGI and 0 otherwise. Go learn. This setting on its own is meaningless! The “how” of the RL agent is not even 99% of the problem, it is all of it. Given our understanding of both DL and neuroscience, it is not even clear to me that we can say with confidence that Neural Networks are a sufficiently expressive architecture to cover an AGI. The h…

> The “how” of the RL agent is not even 99% of the problem, it is all of it

I'm not sure that's true anymore - pretty much any objective devised is being solved by ML solutions within months (with some exceptions such as Chollet's ARC, maybe Winogrande). But those same models will perform poorly on other unseen tasks, because ML takes shortcuts if it can. We used to have unsolved tasks for decades, such as Go. It's now comparably hard (if not harder) to create a good objective measure of intelligence than to reach human parity on said measure.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#119

Earlier quoted context omitted.

Deep Mind is cutting edge ML in general, right? Doesn't Google actively apply the lessons learned all over the place? YouTube content recommendation stands out to me in particular. Translation and automated closed captioning are also obviously ML based. I'd guess that most of the really interesting stuff would be behind the scenes and not immediately visible to end users though.

I’m stressing the business part. YouTube is a loss-making business year after year. Deep Mind gloss doesn’t seem to change that. If indeed it even is Deep Mind making those improvements, Google has lots of other ML groups, such as Google Brain, and these are more directly focused on Google products. There’s no denying their academic success, or game playing etc, but as far as I can see, the data centre cooling bit is…

How do you know that YouTube is a loss making business?

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#120

Everything old is new again. As far as I can tell, they're not actually proposing how to achieve this. I can't access the article without a host institution it seems (is there another link?), so I only have the article to go by. RL has been the basis for all robots engaging with the world, and that engagement with the physical world modeled using RL has been promised to make robots that can act like a 2 year old for…

I don't know who Patricia Churchland is, but they said that the paper was "very carefully and insightfully worked out." After having read the paper, I am very disappointed in the output. Nothing concrete was shown, just hypothesis and reads more like philosophy. That being said, I would say that the paper is carefully worked out and does provide insight if you haven't thought about RL before.

If Patricia Churchland doesn't have a problem with the paper despite it being philosophical, then that's probably because she is a philosopher. An eliminative materialist, to be precise.

Personally, from reading the abstract, I disagree with the hypothesis. There's a trick where anything (even say, a database lookup) looks like optimization as long as you contrive the objective function just right, but that's kind of uninformative.

Post reply on HN