Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

DeepMind says reinforcement learning is ‘enough’ to reach general AI

venturebeat.com

Post reply on HN
Print view View original

271–280 of 312 posts

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#271

Post by goldenkey »

shmageggy wrote: post #239
adipandas wrote: post #182

Earlier quoted context omitted.

I like your take on frame rate of the universe. Nice! :)

Except it's wrong. I recently had the same misconception about the Plank constant somehow being some minimal unit, but it's not. This video from Fermilab's website helped set me straight https://www.youtube.com/watch?v=rzB2R_qiC28

It's not wrong. And the Fermilab video doesn't really dispute it.

Planck's constant measures action, Hz per Joule of energy. Hz is really just a measure of oscillation, or change. It doesn't directly translate to framerate, but it gives us a ballpark figure in orders of magnitude. We don't have anything near 10^34 Hz en-silico, and even if we built a biological/chemical computer, that would be on the par of Avogadro's number, 10^23. So, just because we build a system that can _evolve_ to be intelligent, or hold intelligence within it, doesn't mean we have any ability to actually see it through to that.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#272

Post by state_less »

xamuel wrote: post #265
state_less wrote: post #262

Earlier quoted context omitted.

There are no infinite rewards in biology and yet mathematicians seem to do just fine answering these sorts of questions. I don’t think you want to encode your problem domain in your reward system. It’d be like asking a logic gate to add when you really should be reaching for an FPU. Maybe I’m missing something though?

>There are no infinite rewards in biology and yet mathematicians seem to do just fine answering these sorts of questions This is only a problem if you're already assuming we do everything based on our biological reward systems, and in the current context that would be circular reasoning. Imagine the treasury creates a "superdollar", a product which, if you have one, you can use to create any number of dollars you wan…

> This is only a problem if you're already assuming we do everything based on our biological reward systems

Is there some other way that we are do it beside our biological reward system? It sure looks like we get an apple and not an infinite reward when we pick the right answer to be selecting button B. I understand that might not satisfy you.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#274

Post by throwaway879 »

visarga wrote: post #27
throwaway879 wrote: post #13

If you send a message in a bottle it will eventually land ashore somewhere, maybe in a century, who knows, and who knows whether it will be relevant by then or not, or civilization may not even exist by then, but sure it's similarly plausible to get to AGI via RL.

> send a message in a bottle Just one - yes. But how about if you send millions of bottle messages?

>>how about if you send millions of bottle messages?

assuming we can integrate all learnings from those bottles into a system that can classify any given situation and apply the learning in that domain. But to build a system that can classify any problem is where we're stuck at and RL can't get us there

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#275

Post by enkid »

xamuel wrote: post #236
enkid wrote: post #227

Earlier quoted context omitted.

How do you even have reinforcement learning with non-real numbers? The point is to maximize a score. It seems to me, any benefit you'd get from using an alternative number system could be replicated by using an algorithm to convert multiple real number scores into a single value.

Here's an example. Suppose there are two buttons, A and B. If you press A for the nth time, then you get reward n. If you press B for the nth time, then you get reward 0 if n is not a power of 2, or reward omega (the first infinite ordinal number) if n is a power of 2. If the above rewards are shoehorned into real numbers---for example, by replacing omega with 9999 or something---then an RL agent would misunderstand…

What's the behavior your trying to get the AI to do in this example? Learn how to compute the power of 2? This is a task that can be accomplished much more simply with a different reward system. For example, have A always equal 1 and B equal 2 if it is a power if 2 and 0 otherwise.

I understand you can use non real numbers, that's not what I was asking. I'm asking what's a behaviour you can't replicate using a reward system based on real numbers.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#276

Post by prometheus76 »

neltnerb wrote: post #4
webmaven wrote: post #3

Saying RL is sufficient to (eventually) achieve AGI is a bit misleading. One might similarly state that biological evolution is sufficient to (eventually) achieve biological general intelligence. Both statements are probably true, but the parenthetical (eventually) is doing an awful lot of heavy lifting.

Yes, it's easy to be convinced on either side, the arguments write themselves. Yes, eventually a learning system might learn enough to be indistinguishable from intelligence. Or this might be entirely the wrong path and detracting from genuine new innovations in how we think about AI. We won't be able to tell whether it's AGI or just good enough at trained tasks to trick us.

Let's say I'm standing next to a table. The computer recognizes it as a table. Now I sit on the table. Is it a chair or a table now? Something that we do automatically is a LONG way away from being automatic for AI.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#277

Post by cowanon22 »

The AI field has a history of over confidence; just look up Marvin Minsky's predications in the 1950's and 1960's. This had been repeated throughout AI history - a new approach is found, it has some promising initial results, and then progress gets stuck and the approach is basically halted for decades.

Machine learning and neural networks have prospered due to massive increases in computing speed and data, but I suspect it will also be a dead end in general AI. There will be some niche wins, some quite impressive, but the machine won't have the properties of an intelligent child or even a rat.

There seems to be a big difference in how actual neurons work - I think neural networks are misnamed. The brain is not a big matrix, biology does not appear to reinforce behavior using gradient operations. I think increased research into neuroscience and biomimicry is the most likely approach for general AI, but I think we are still far away and the current hype will just be one more dead end.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#278

Post by lupire »

Veedrac wrote: post #219
burning_hamster wrote: post #137

Earlier quoted context omitted.

I think the title of the paper makes more sense if you consider that ten years ago, someone could have written a paper in a similar spirit with a different take on "what is enough". Back then, it would probably have been titled: "Backpropagation of errors is enough". The last ten years have shown that backpropagation -- while a crucial component -- is not enough. Personally, I would not be shocked to find out in the…

Ten years ago we didn't even have AlexNet; I think most people would have thought a paper like that was nuts at the time. The ten years since are what popularized backpropagation as a path to general intelligence. Who ten years ago would have seriously predicted GPT-3? The odd few that did are certainly not the people I would expect to have been dissuaded! And if there's any actual experimental evidence that backprop…

Backpropagation was the model for AGI in the 1980s if not earlier. Of course computing power made it impossible for anything to actually deliver AGI.

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#279

Post by lupire »

ABCLAW wrote: post #220
endtime wrote: post #192

Earlier quoted context omitted.

Why are forgetting and sleep relevant? If someone invented a pill that gave you a perfect memory and removed the need to sleep, would you stop being generally intelligent if you took it?

One of the postulated reasons for why older people have worse reaction times and think slower than their younger counterparts is that the neural networks they use draw upon more stored information, thus making routine evaluations take longer. There's a sweet spot between knowing enough and knowing little enough so that you get the right answer and get it quickly enough.

That's a weird claim. Why not just assume old people are slower minded for the same reason they are physically slower: physical degredation?

Re: DeepMind says reinforcement learning is ‘enough’ to reach general AI

#280

Post by Viliam1234 »

ikrenji wrote: post #198
admk wrote: post #114

Earlier quoted context omitted.

That is if you believe biological general intelligence is the end goal of evolution, which I believe is highly unlikely. Intelligence is simply a special side-product of evolution, there is nothing general about general intelligence. Many organisms can thrive without it. There is also a non-negligible chance that all organisms would die out before reaching intelligence. We are fortunate to live in a world that produc…

i mean if the end goal is to propagate the organism, surely intelligence will be helpful to this - interplanetary scale

But until that actually happens, the possibility of it maybe happening in the future has zero impact on current natural selection.
Post reply on HN