Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind says reinforcement learning is ‘enough’ to reach general AI

venturebeat.com

Post reply on HN
Print view View original

231–240 of 312 posts

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#231

Post by Filligree »

didericis wrote: post #217
Filligree wrote: post #165

Earlier quoted context omitted.

> That is if you believe biological general intelligence is the end goal of evolution, which I believe is highly unlikely. I would agree, but might add that evolution doesn't have 'goals'. Is that the point you were trying to make?

Not OP, but yeah, evolution doesn’t have goals in the same sense that people do, just like gravity doesn’t “want” to pull things, it just kind of “is”, and simply acts as reality permits based on prior and current conditions. That’s reasonable to say. Convergent evolution exists for at least some adaptations though, like the eye. It’s not unreasonable to think that there may be some sort of equivalent convergence whi…

Ah. So that’s why we exist. I was wondering.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#232

Post by mensetmanusman »

Does anyone know if deepmind is working on driverless technology?

It seems like one of the largest values they could bring to society would be to solve driverless technology.

Currently their best partner would be Tesla due to the amount of data Tesla has, but I doubt Google would allow that collaboration…

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#233

Post by xamuel »

robrenaud wrote: post #225
xamuel wrote: post #213

Earlier quoted context omitted.

I thought it was a fun position paper, if not exactly groundbreaking. They did avoid one common pitfall at least. They are (intentionally?) vague about which number systems the rewards can come from, apparently leaving it open whether the rewards need be real-valued or whether they can be, say, hyperreals, surreals, computable ordinals, etc. This avoids a trap I've written about elsewhere [1]: traditionally, RL rewar…

There are more real numbers than programs. Computers cannot represent the vast majority of real numbers. AFAICT, it's not even clear that the universe is continuous rather than discrete. I really don't believe that using approximations of real numbers is going to be the bottleneck for AGI.

And yet computers have no problem symbolically representing non-rational numbers like sqrt(2), pi, etc. Neither is there any inherent reason why they cannot symbolically represent various levels of infinity, nor why those would be incomprehensible to AGIs (even if the universe is discrete). You're right that only countably many numbers can be represented, but nevertheless even countable subsets of extended number systems can exhibit structural properties that the reals do not exhibit.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#234

Post by X6S1x6Okd1st »

mensetmanusman wrote: post #232

Does anyone know if deepmind is working on driverless technology? It seems like one of the largest values they could bring to society would be to solve driverless technology. Currently their best partner would be Tesla due to the amount of data Tesla has, but I doubt Google would allow that collaboration…

https://www.businessinsider.com/deepmind-is-teaching-googles...

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#235

Post by eatmyshorts »

mensetmanusman wrote: post #232

Does anyone know if deepmind is working on driverless technology? It seems like one of the largest values they could bring to society would be to solve driverless technology. Currently their best partner would be Tesla due to the amount of data Tesla has, but I doubt Google would allow that collaboration…

I'm curious why you feel their best partner would be Tesla instead of Alphabet's own Waymo?

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#236

Post by xamuel »

enkid wrote: post #227
xamuel wrote: post #213

Earlier quoted context omitted.

I thought it was a fun position paper, if not exactly groundbreaking. They did avoid one common pitfall at least. They are (intentionally?) vague about which number systems the rewards can come from, apparently leaving it open whether the rewards need be real-valued or whether they can be, say, hyperreals, surreals, computable ordinals, etc. This avoids a trap I've written about elsewhere [1]: traditionally, RL rewar…

How do you even have reinforcement learning with non-real numbers? The point is to maximize a score. It seems to me, any benefit you'd get from using an alternative number system could be replicated by using an algorithm to convert multiple real number scores into a single value.

Here's an example. Suppose there are two buttons, A and B. If you press A for the nth time, then you get reward n. If you press B for the nth time, then you get reward 0 if n is not a power of 2, or reward omega (the first infinite ordinal number) if n is a power of 2.

If the above rewards are shoehorned into real numbers---for example, by replacing omega with 9999 or something---then an RL agent would misunderstand the environment and would eventually be misled into thinking that pressing A yields more average reward.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#237

Post by robertlagrant »

virgilp wrote: post #31
randcraw wrote: post #8

Earlier quoted context omitted.

Like the notion that all of natural language can be modeled fully if you just use enough finite state machines — surely true; just a wee bit inefficient.

Wait no, that can't be true. Finite state machines are only as powerful as regular expressions; you need pushdown automata even for programming languages, surely you can't model natural language with finite state machines? (maybe an infinite number of them, but I'm doubtful even on that - I'd have to review the theory to be sure though)

Perhaps the fact that some words can never follow other words would make it tractable.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#238

Post by exo-pla-net »

LesZedCB wrote: post #222

the paper postulates that "reward is enough" but what is the alternative? rewards plus what? the "cybernetic feedback loop" aka reinforcement learning has kinda always been the idea, no? i believe that architecture is important. yes, at one level the human is a rewards machine, but the neocortex produces a lot of "self-reflection" that doesn't really generate reward in itself, at least i don't know if it does. maybe…

One needs to actually read the paper to understand what it's postulating; the paper's title is not enough.

There is a temptation to view our capabilities as a complex set of interacting modules. Under that view, each module would need to be separately developed in order to artificially recreate our abilities. The paper makes the case that we need only a single carrot.

Per the paper:

"For example, the ability of social intelligence has often been framed as the Nash equilibrium of a multi-agent system; the ability of language by a combination of goals such as parsing, part-of-speech tagging, lexical analysis, and sentiment analysis; and the ability of perception by object segmentation and recognition.

In this paper, we consider an alternative hypothesis: that the generic objective of maximising reward is enough to drive behaviour that exhibits most if not all abilities that are studied in natural and artificial intelligence."

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#239

Post by shmageggy »

adipandas wrote: post #182
goldenkey wrote: post #7

Earlier quoted context omitted.

Intelligence would be produced in any Turing complete automata. But the universe has a frame rate of 10^34 (based on Planck constant.) We don't really have the tech to just run "evolution" of a universe or of even a psuedo biological substrate.

I like your take on frame rate of the universe. Nice! :)

Except it's wrong. I recently had the same misconception about the Plank constant somehow being some minimal unit, but it's not. This video from Fermilab's website helped set me straight https://www.youtube.com/watch?v=rzB2R_qiC28

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#240

Post by bobthechef »

rich_sasha wrote: post #110

Basically, any problem with a solution fits into RL: reward of 1 if you are AGI and 0 otherwise. Go learn. This setting on its own is meaningless! The “how” of the RL agent is not even 99% of the problem, it is all of it. Given our understanding of both DL and neuroscience, it is not even clear to me that we can say with confidence that Neural Networks are a sufficiently expressive architecture to cover an AGI. The h…

Concepts can't be represented in matter. That's your secret sauce. Well, the beginning of the recipe, anyway. But you won't be able to make the dish.
Post reply on HN