Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Bitter Lesson (2019)

incompleteideas.net

Post reply on HN
Print view View original

51–60 of 87 posts

Re: The Bitter Lesson (2019)

#51

Post by burntoutfire »

YeGoblynQueenne wrote: post #34

>> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. "Massive, deep search" that started from a book of opening moves and the combined expert knowledge of several chess Grandmasters. And that was an instance of the minimax algorithm with alpha-beta cutoff, i.e. a search algorithm specifically designed for two-player, deterministic games like chess.…

> No, no. The biggest question in the field is not one that is answered by "a deeper search". The biggest question is "how can we do that without a search"?

My guess is that we're doing pattern recognision, where we recognize taht a current game state is similar to a situation that we've been in before (in some previous game), and recall the strategy we took and the outcomes it had lead to. With large enough body of experience, you can to remember lots of past attempted strategies for every kind of game state (of course, within some similarity distance).

Re: The Bitter Lesson (2019)

#52

Post by blt »

burntoutfire wrote: post #51
YeGoblynQueenne wrote: post #34

>> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. "Massive, deep search" that started from a book of opening moves and the combined expert knowledge of several chess Grandmasters. And that was an instance of the minimax algorithm with alpha-beta cutoff, i.e. a search algorithm specifically designed for two-player, deterministic games like chess.…

> No, no. The biggest question in the field is not one that is answered by "a deeper search". The biggest question is "how can we do that without a search"? My guess is that we're doing pattern recognision, where we recognize taht a current game state is similar to a situation that we've been in before (in some previous game), and recall the strategy we took and the outcomes it had lead to. With large enough body of…

This insight is the essence of the AlphaZero architecture. Whereas a pure Monte Carlo Tree Search (MCTS) starts each node in the search tree with a uniform distribution over actions, AlphaZero trains a neural network to observe the game state and output a distribution over actions. This distribution is optimized to be as similar as possible to the distribution obtained from running MCTS from that state in the past. It's very similar to the way humans play games.

Re: The Bitter Lesson (2019)

#53

Post by YeGoblynQueenne »

dreamcompiler wrote: post #43
YeGoblynQueenne wrote: post #34

>> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. "Massive, deep search" that started from a book of opening moves and the combined expert knowledge of several chess Grandmasters. And that was an instance of the minimax algorithm with alpha-beta cutoff, i.e. a search algorithm specifically designed for two-player, deterministic games like chess.…

Are we certain that well-trained human players are not doing search? It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

>> Are we certain that well-trained human players are not doing search?

Yes- because human players can only search a tiny portion of a game tree and a minimax search of the same extent is not even sufficient to beat a dedicated human in tic-tac-to, leta lone chess. That is, unless one wishes to countenance the possibility of an "unconscious search" which of course might as well be "the grace of God" or any such hand-wavy non-explanation.

>> It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

Sorry, I don't understand what you mean.

Re: The Bitter Lesson (2019)

#54

Post by YeGoblynQueenne »

gwern wrote: post #45
dreamcompiler wrote: post #43

Earlier quoted context omitted.

Are we certain that well-trained human players are not doing search? It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

I'm not sure why YeGoblynQueenne thinks this is such a mystery. (This is not the first time I've been puzzled by their pessimism on HN.) There is no mystery here: AlphaZero shows that you can get superhuman performance by searching only a few ply by sufficiently good pattern recognition in a highly parameterized and well-trained value function, and MuZero makes this point even more emphatically by doing away with the…

MuZero performs a "formal search". In many more ways than one, for example optimisation is still a search for an optimal search of parameters. But I guess you mean that it doesn't perform a tree search? Quoting from the abstract of the paper on arxiv [1]:

In this work we present the MuZero algorithm which, by combining _a tree-based search_ with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, withoutany knowledge of their underlying dynamics.

(My underlining)

If I remember correctly, MuZero is model-free in the sense that it learns its own evaluation function and reward policy etc (also going by the abstract). But it retains MCTS.

Indeed, it wouldn't really make sense to drop MCTS from the architecture of a system designed to play games. I mean, it would be really hard to justify discarding a component that is well known to work and work well, both from an engineering and a scientific point of view.

_________________

https://arxiv.org/abs/1911.08265

Re: The Bitter Lesson (2019)

#55

Post by YeGoblynQueenne »

new2628 wrote: post #46
YeGoblynQueenne wrote: post #34

>> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. "Massive, deep search" that started from a book of opening moves and the combined expert knowledge of several chess Grandmasters. And that was an instance of the minimax algorithm with alpha-beta cutoff, i.e. a search algorithm specifically designed for two-player, deterministic games like chess.…

At least in chess, if it is not the search, then it is probably the evaluation function. Expert players have likely a very well-tuned evaluation function of how strong a board "feels". Some of it is explainable easily: center domination, diagonal bishop, connected pawn structure, rook supporting pawn from behind, others are more elaborate, come with experience and harder to verbalize. When expert players play against…

I like to caution against taking concepts from computer science and AI and applying them directly to the way the human mind works. Unless we know that a player is applying a specific evaluation function (e.g. because they tell us, or because they vocalise their thought process etc) then even suggesting that "players have an evaluation function" is extrapolating far from what it is safe. For one thing- what does a "function" look like in the human mind?

Whatever human minds do, computing is only a very general metaphor for it and it's very risky to assume we understand anything about our mind just because we understand our computers.

Re: The Bitter Lesson (2019)

#56

Post by YeGoblynQueenne »

gwern wrote: post #45
dreamcompiler wrote: post #43

Earlier quoted context omitted.

Are we certain that well-trained human players are not doing search? It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

I'm not sure why YeGoblynQueenne thinks this is such a mystery. (This is not the first time I've been puzzled by their pessimism on HN.) There is no mystery here: AlphaZero shows that you can get superhuman performance by searching only a few ply by sufficiently good pattern recognition in a highly parameterized and well-trained value function, and MuZero makes this point even more emphatically by doing away with the…

>> (This is not the first time I've been puzzled by their pessimism on HN.)

I don't understand why you keep making personal comments like that about me. I suspect you don't realise that they are unpleasant. Please let me make it clear: such personal comments are unpleasant. Could you please stop them? Thank you.

Re: The Bitter Lesson (2019)

#57

Post by oezi »

YeGoblynQueenne wrote: post #53
dreamcompiler wrote: post #43

Earlier quoted context omitted.

Are we certain that well-trained human players are not doing search? It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

>> Are we certain that well-trained human players are not doing search? Yes- because human players can only search a tiny portion of a game tree and a minimax search of the same extent is not even sufficient to beat a dedicated human in tic-tac-to, leta lone chess. That is, unless one wishes to countenance the possibility of an "unconscious search" which of course might as well be "the grace of God" or any such hand-…

Why do you dismiss the unconscious search that humans do in Go? Having learned Go some years ago it is such an exciting thing to realize that with practice the painstaking process of consciously evaluating the myriads possibilities of moves gives way to just "seeing" solutions out of nothing. You can really feel that your brain did wire itself up to do analysis for you at a level that is subconscious but interfaces so gracefully with your conscious cognition that it is a real marvel.

Re: The Bitter Lesson (2019)

#58

Post by YeGoblynQueenne »

maest wrote: post #13

For contrast, take this Hofstadter quote: > This, then, is the trillion-dollar question: Will the approach undergirding AI today—an approach that borrows little from the mind, that’s grounded instead in big data and big engineering—get us to where we want to go? How do you make a search engine that understands if you don’t know how you understand? Perhaps, as Russell and Norvig politely acknowledge in the last chapte…

Where is this quote from, please?

To play devil's advocate, I think retort to your comment about "intellectually satisfying" methods is "yeah, but, they work". And in any case, "intellectually satisfying" doesn't have a formal definition in computer science or AI so it can't very well be a goal, as such.

My own concern is exactly what Russel & Norvig seem to say in Hofstadter's comment: by spending all our resources on clmbing the tallest trees to get to the moon, we're falling behind from our goal, of ever getting to the moon. That's even more so if the goal is to use AI to understand our own mind, rather than to beat a bunch of benchmarks.

Re: The Bitter Lesson (2019)

#59

Post by YeGoblynQueenne »

oezi wrote: post #57
YeGoblynQueenne wrote: post #53

Earlier quoted context omitted.

>> Are we certain that well-trained human players are not doing search? Yes- because human players can only search a tiny portion of a game tree and a minimax search of the same extent is not even sufficient to beat a dedicated human in tic-tac-to, leta lone chess. That is, unless one wishes to countenance the possibility of an "unconscious search" which of course might as well be "the grace of God" or any such hand-…

Why do you dismiss the unconscious search that humans do in Go? Having learned Go some years ago it is such an exciting thing to realize that with practice the painstaking process of consciously evaluating the myriads possibilities of moves gives way to just "seeing" solutions out of nothing. You can really feel that your brain did wire itself up to do analysis for you at a level that is subconscious but interfaces s…

>> Why do you dismiss the unconscious search that humans do in Go?

The question is why you say that humans perform an unconscious search when they play Go. And what kind of search is it, other than unconscious? Could you describe it, e.g. in algorithmic notation? I mean, I'm sure you couldn't because if you could then the problem of teaching a computer to play Go as well as a human would have been solved years and years ago. But, if you can't describe what you're doing, then how do you know it's a "search"?

Note that in AI, when we talk of "search" (edit: at least, in the context of game-playing) we mean something very specific: an algorithm that examines the nodes of a tree and applies some criterion to label each examined node as a target node or not a target node. Humans are absolutely awful at executing such an algorithm with our minds for any but the most trivial of trees, at least compared to computers.

Re: The Bitter Lesson (2019)

#60

Post by mtgp1000 »

YeGoblynQueenne wrote: post #34

>> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. "Massive, deep search" that started from a book of opening moves and the combined expert knowledge of several chess Grandmasters. And that was an instance of the minimax algorithm with alpha-beta cutoff, i.e. a search algorithm specifically designed for two-player, deterministic games like chess.…

>But, rather than an irrelevant "bitter" lesson about how big machines can perfom more computations than a human, a really useful lesson -and one that we haven't yet learned, as a field- is why humans can do so well without search

I think the answer is heuristics based on priors(e.g. board state), which we've demonstrated (with alphago and derivatives, especially alphago zero) that neural networks are readily able to learn.

This is why I get the impression that modern neural networks are quickly approaching humanlike reasoning - once you figure out how to

(1) encode (or train) heuristics and

(2) encode relationships between concepts in a manner which preserves a sort of topology (think for example of a graph where nodes represent generic ideas)

You're well on your way to artificial general reasoning - the only remaining question becomes one of hardware (compute, memory, and/or efficiency of architecture).

Post reply on HN