Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Google DeepMind Means for A.I.

newyorker.com

Post reply on HN
Print view View original

61–70 of 94 posts

Re: What Google DeepMind Means for A.I.

#61

Post by pixl97 »

Retric wrote: post #44
Animats wrote: post #31

This is impressive. The current approach will work only for games where the whole state is on-screen and planning isn't required. A pure reactive system will work for that. I used to say that a key component of AI that was missing was the ability to get through the next few seconds of life without falling down or bumping into anything. I went through Stanford CS when the top-down logicians were in charge of AI. That…

These low level approaches can do planning, there simply ridiculously inefficient at it. A lot of AI research is focused on the idea that 1 Trillion floating point operations per second on 1,000,000,000 bytes of data is now cheap. Efficiency is simply less important.

>Efficiency is simply less important.

"More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason — including blind stupidity." — W.A. Wulf

Re: What Google DeepMind Means for A.I.

#62

Post by dragonwriter »

NamTaf wrote: post #60

For all of the New Yorker's clout in journalism, sentences like the following make me wonder where their editors are. The run-on and sea of commas is atrocious! It's not the first time I've noticed this in the last few days either. "Hassabis, who began working as a game designer in 1994, at the age of seventeen, and whose first project was the Golden Joystick-winning Theme Park, in which players got ahead by, among o…

That's not a run-on, and all the commas are properly used. Its stylistically awful because there are way to many apositive/parenthetical phrases getting in the way, and because they are nested without using an alternative device (like setting the outer one off with dashes rather than commas) as well as pointless excess verbosity ("at least five years away from being a decade behind the gaming curve").

Re: What Google DeepMind Means for A.I.

#63

Post by zhanwei »

fitzwatermellow wrote: post #39
karpathy wrote: post #17

I really like this line of work and I expect will grow quite substantially over the next few years. Of course, Reinforcement Learning has been around for a long time. Similarly, Q Learning (the core model in this paper) has been around a very long time. What is new is that normally you see these models applied to toy MDP problems with simple dynamics, and linear Q function approximations for fear of non-convergence e…

Thanks, karpathy! I did submit your JS implementation to HN when I came across it: https://news.ycombinator.com/item?id=9108738 Monte Carlo Tree Search could be the missing link. In other words, use DQN to model the world and map actions to a value function. Then use playouts and backpropogation of action tree results to find tactics. Of course, it does not solve the big question: how to model "memories" and "inferen…

We need a model to use Monte Carlo Tree Search... which is missing in this approach since it uses model-free reinforcement learning. Unless the deep convolution net can extract some latent features as state, it would be impossible to do planning on top of it.

Re: What Google DeepMind Means for A.I.

#64

Post by malkia »

Houshalter wrote: post #34
Animats wrote: post #31

This is impressive. The current approach will work only for games where the whole state is on-screen and planning isn't required. A pure reactive system will work for that. I used to say that a key component of AI that was missing was the ability to get through the next few seconds of life without falling down or bumping into anything. I went through Stanford CS when the top-down logicians were in charge of AI. That…

I remember an AI researcher (I forget who) recently said something to the effect that early AI research produced all sorts of planning algorithms. E.g. the top down camp of AI. But they weren't capable of working with the real world because we didn't have very good low level perception. E.g. this complicated algorithm for planning the robot's actions, but it depended on getting input about where objects are. Now we h…

Well, it's basically the same with me and reading math expressions filled with symbols out of nowhere, and you need to know the context, history and how they are used... If someone translated this to me as lisp/scheme or some other form of symbols, with full-meaningful names I might have better idea of what's going on.

Okay, not exactly the same... but won't mind AI algorithm that sorts it out for me.

Re: What Google DeepMind Means for A.I.

#65

Post by chockablock »

_abattoir wrote: post #50
chockablock wrote: post #37

Earlier quoted context omitted.

>We experience the world at one second per second There is evidence from animal studies that the hippocampus (a brain structure critical for memory) can 'replay' remembered events at 10-20x speedup. See, for example: http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ Video at: http://youtu.be/Bv7zN2Or6Mg (Full-disclosure: I am the first author.) And in fact the OP uses biologically-inspired off-line replay as part of the…

Fascinating. Does this relate to the speed of dreams? i.e. a "dream" might seem to have taken hours when in fact the REM sequence was on the order of seconds?

Could do--the phenomenon was first seen in sleeping animals. Replay during REM is probably at 1X speed, non-REM (aka slow wave sleep) is in the 10-20x range. The dreams you remember (long, emotionally involved) tend to happen in REM. Of course, we can't ask rats what they dreamt about, but the patterns in the neural activity are very specific (we can decode the rat's position with ~5cm accuracy, and during replay we see coherent trajectories through space that look just like real movement.)

Also Hassabis knows this literature well--he has published work on decoding human memories in hippocampus from fMRI data and we talked shop about replay a long time ago. (Very thoughtful and friendly person FWIW).

Re: What Google DeepMind Means for A.I.

#66

Post by NamTaf »

dragonwriter wrote: post #62
NamTaf wrote: post #60

For all of the New Yorker's clout in journalism, sentences like the following make me wonder where their editors are. The run-on and sea of commas is atrocious! It's not the first time I've noticed this in the last few days either. "Hassabis, who began working as a game designer in 1994, at the age of seventeen, and whose first project was the Golden Joystick-winning Theme Park, in which players got ahead by, among o…

That's not a run-on, and all the commas are properly used. Its stylistically awful because there are way to many apositive/parenthetical phrases getting in the way, and because they are nested without using an alternative device (like setting the outer one off with dashes rather than commas) as well as pointless excess verbosity ("at least five years away from being a decade behind the gaming curve").

When I say run-on I guess I mean the sentence just drags on when it could be two or three separate sentences with no negative impact. It makes it difficult to read and I, at least, lose track of where I am in it.

It's not incorrect per se, it's just not well-written as far as I'm concerned. But what do I know, I'm not a journalist.

Re: What Google DeepMind Means for A.I.

#67

Post by kriro »

chockablock wrote: post #37
iandanforth wrote: post #4

"video games" (read "world simulator"). The important thing about their work is that it is deliberately marching down the path of more and more complex world simulations. We experience the world at one second per second. To learn to walk we must first fall, and we fall at 32 feet/second^2. There's a hard limit on how fast we can make mistakes (like tripping) and so there is a hard limit on how fast we can learn. Comp…

>We experience the world at one second per second There is evidence from animal studies that the hippocampus (a brain structure critical for memory) can 'replay' remembered events at 10-20x speedup. See, for example: http://www.ncbi.nlm.nih.gov/m/pubmed/19709631/ Video at: http://youtu.be/Bv7zN2Or6Mg (Full-disclosure: I am the first author.) And in fact the OP uses biologically-inspired off-line replay as part of the…

This is pretty interesting (the high level conclusion, I do not claim to understand more). I've recently done a bit of pretty light research on the sense of smell (in the context of AR/VR) and it seems like smell can "transport you across time/space" faster than other senses. Might be an interesting follow up. If one can speed up recent events for quick recall/learning naturally maybe smell can somehow be used to speed up the going back in time aspect (some smell trigger to make the event feel recent+the 10-20x speedup combined)?

Re: What Google DeepMind Means for A.I.

#68

Post by kriro »

I'm only loosely familiar with the "general game playing" literature but I think learning the structure of the game is the interesting research problem here. An immediate experiment that I'd like to try would be:

(a) Identify games considered similar (+maybe define what it means to be similar)...let's take Pong and Breakout as suggested (b) Trial and error run of one of the games to learn the action-reward structure (c) Compare a fresh relearning of the second game to a start where the similarities/differences are pre-input to the AI "somehow" (some sort of diff between the game rules etc.)

I'm thinking of a gamer thinking "oh this is just like X,Y,Z except..." when picking up a new game.

Re: What Google DeepMind Means for A.I.

#69

Post by rwallace »

jacquesm wrote: post #5
iandanforth wrote: post #4

"video games" (read "world simulator"). The important thing about their work is that it is deliberately marching down the path of more and more complex world simulations. We experience the world at one second per second. To learn to walk we must first fall, and we fall at 32 feet/second^2. There's a hard limit on how fast we can make mistakes (like tripping) and so there is a hard limit on how fast we can learn. Comp…

For now real-world simulations at the complexity level of 'a toddler' run at a very small fraction of real time.

Or to be more exact: if we could build such things at all, which we can't yet, they would run at a very small fraction of real-time.

Re: What Google DeepMind Means for A.I.

#70

Post by mjrpes »

NamTaf wrote: post #60

For all of the New Yorker's clout in journalism, sentences like the following make me wonder where their editors are. The run-on and sea of commas is atrocious! It's not the first time I've noticed this in the last few days either. "Hassabis, who began working as a game designer in 1994, at the age of seventeen, and whose first project was the Golden Joystick-winning Theme Park, in which players got ahead by, among o…

Sentences like this are kind of the New Yorker's erudite (you could argue pretentious) style. It works better on the the printed page than on the web. If you read the New Yorker often enough your mind gets used to it and it isn't too much of a bother. Granted, this sentence would have been better off with a period in there somewhere.... it's a bit overkill.
Post reply on HN