Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Bitter Lesson (2019)

incompleteideas.net

Post reply on HN
Print view View original

71–80 of 87 posts

Re: The Bitter Lesson (2019)

#71

Post by fxtentacle »

YeGoblynQueenne wrote: post #53
dreamcompiler wrote: post #43

Earlier quoted context omitted.

Are we certain that well-trained human players are not doing search? It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

>> Are we certain that well-trained human players are not doing search? Yes- because human players can only search a tiny portion of a game tree and a minimax search of the same extent is not even sufficient to beat a dedicated human in tic-tac-to, leta lone chess. That is, unless one wishes to countenance the possibility of an "unconscious search" which of course might as well be "the grace of God" or any such hand-…

I believe humans do look up tables. If pattern X, then deny possibilities Y and Z. And then you more or less consciously choose one of the remaining options.

If you can avoid making "obvious" (in retrospect) mistakes, you will be a competitive chess or go player.

And given that we have millions of neurons to suppress activity in one node if activity was high enough in another node, I would summarize the brain as effectively a huge chain of look up tables.

Re: The Bitter Lesson (2019)

#72

Post by YeGoblynQueenne »

self wrote: post #63
YeGoblynQueenne wrote: post #58

Earlier quoted context omitted.

Where is this quote from, please? To play devil's advocate, I think retort to your comment about "intellectually satisfying" methods is "yeah, but, they work". And in any case, "intellectually satisfying" doesn't have a formal definition in computer science or AI so it can't very well be a goal, as such. My own concern is exactly what Russel & Norvig seem to say in Hofstadter's comment: by spending all our resources…

The quote is from this article: https://www.theatlantic.com/magazine/archive/2013/11/the-man...

Thank you.

Gosh, what an awkward pose, that first picture.

Re: The Bitter Lesson (2019)

#73

Post by YeGoblynQueenne »

francoisp wrote: post #36

building a model for and with domain knowledge == premature optimization? In the end a win on kaggle or a published paper seems to depend on tweaking hyperparameters based on even more pointed DK: data set knowledge... I wonder what would be required to build a model that explores the search space of compilable programs in say python that sorts in correct order. Applying this idea of using ML techniques to finding be…

>> I wonder what would be required to build a model that explores the search space of compilable programs in say python that sorts in correct order.

Oh, not that much. You could do that easily with a small computer and an infinite amount of time.

Re: The Bitter Lesson (2019)

#74

Post by acidbaseextract »

YeGoblynQueenne wrote: post #59
oezi wrote: post #57

Earlier quoted context omitted.

Why do you dismiss the unconscious search that humans do in Go? Having learned Go some years ago it is such an exciting thing to realize that with practice the painstaking process of consciously evaluating the myriads possibilities of moves gives way to just "seeing" solutions out of nothing. You can really feel that your brain did wire itself up to do analysis for you at a level that is subconscious but interfaces s…

>> Why do you dismiss the unconscious search that humans do in Go? The question is why you say that humans perform an unconscious search when they play Go. And what kind of search is it, other than unconscious? Could you describe it, e.g. in algorithmic notation? I mean, I'm sure you couldn't because if you could then the problem of teaching a computer to play Go as well as a human would have been solved years and ye…

Here's a section of Michael Redmond's (9-dan professional Go player) commentary on the Lee Sedol vs AlphaGo matches: https://youtu.be/yCALyQRN3hw?t=3031

It's really fun to watch his commentary because he relentlessly plays "variations" — possible next moves and sequences — while waiting for the players, explaining the tradeoffs between moves and the consequences they lead to a few steps ahead in the game.

I don't know what to call "variations" but a tree search with heuristics. He does it slowly to explain it to the audience, but I have no doubt the same process runs much faster in his mind.

Re: The Bitter Lesson (2019)

#75

Post by gwern »

YeGoblynQueenne wrote: post #54
gwern wrote: post #45

Earlier quoted context omitted.

I'm not sure why YeGoblynQueenne thinks this is such a mystery. (This is not the first time I've been puzzled by their pessimism on HN.) There is no mystery here: AlphaZero shows that you can get superhuman performance by searching only a few ply by sufficiently good pattern recognition in a highly parameterized and well-trained value function, and MuZero makes this point even more emphatically by doing away with the…

MuZero performs a "formal search". In many more ways than one, for example optimisation is still a search for an optimal search of parameters. But I guess you mean that it doesn't perform a tree search? Quoting from the abstract of the paper on arxiv [1]: In this work we present the MuZero algorithm which, by combining _a tree-based search_ with a learned model, achieves superhuman performance in a range of challengi…

MuZero doesn't do a search over explicit board states, MCTS (where are the playouts at the leaf nodes? where is the simulator state?) or otherwise: it does search over internal-abstract-reward-predictive-states/action pairs, much like a human does, who thinks through possible actions using an internal representation sometimes backtracking as they decide a move is bad and evaluating a different one which feels better. It's search over a tree of possible actions, sure, but this is not MCTS, even if they loosely use the phrase in places (similarly, the tree search Zero does for gameplay is not MCTS, even if people sometimes describe it that way or conflate it with the training).

Re: The Bitter Lesson (2019)

#77

Post by bsder »

dreamcompiler wrote: post #43
YeGoblynQueenne wrote: post #34

>> In computer chess, the methods that defeated the world champion, Kasparov, in 1997, were based on massive, deep search. "Massive, deep search" that started from a book of opening moves and the combined expert knowledge of several chess Grandmasters. And that was an instance of the minimax algorithm with alpha-beta cutoff, i.e. a search algorithm specifically designed for two-player, deterministic games like chess.…

Are we certain that well-trained human players are not doing search? It's possible that a search subnetwork gets "compiled without debugger symbols" and the owner of the brain is simply unaware that it's happening.

> Are we certain that well-trained human players are not doing search?

Some, but there's a LOT more context pruning the search space.

Watch some of the chess grandmasters play and miss obvious winning moves. Why? "Well, I didn't bother looking at that because doesn't just hang a rook randomly."

Re: The Bitter Lesson (2019)

#78

Post by YeGoblynQueenne »

gwern wrote: post #75
YeGoblynQueenne wrote: post #54

Earlier quoted context omitted.

MuZero performs a "formal search". In many more ways than one, for example optimisation is still a search for an optimal search of parameters. But I guess you mean that it doesn't perform a tree search? Quoting from the abstract of the paper on arxiv [1]: In this work we present the MuZero algorithm which, by combining _a tree-based search_ with a learned model, achieves superhuman performance in a range of challengi…

MuZero doesn't do a search over explicit board states, MCTS (where are the playouts at the leaf nodes? where is the simulator state?) or otherwise: it does search over internal-abstract-reward-predictive-states/action pairs, much like a human does, who thinks through possible actions using an internal representation sometimes backtracking as they decide a move is bad and evaluating a different one which feels better.…

I'm a bit confused. MuZero doesn't do a "formal search" (as per your previous comment) but it does a tree search (as per your current comment). It doesn't perform MCTS, even though the paper itself states it performs MCTS.

I quote from the arxiv paper again:

Appendix B Search

We now describe the search algorithm used by MuZero. Our approach is based upon Monte-Carlo tree search with upper confidence bounds, an approach to planning that converges asymptotically to the optimal policy in single agent domains and to the minimax value function in zero sum games [22].

So I'm sorry but I really don't understand what you mean here.

Also, as discussed in other comments, it's dangerous to assume anyhing about how "a human does" anything to do with any kind of mental calculation. Whatever MuZero does and however good or bad it does it, there is nothing to tell us that it does it as a human does.

Re: The Bitter Lesson (2019)

#79

Post by overhyp »

I would like to offer what I believe is a counterpoint, but I am not a trained ML researcher so I am not sure if it is even a counter-point. Maybe it is just an observation.

I recently participated in the following Kaggle competition:

https://www.kaggle.com/allen-institute-for-ai/CORD-19-resear...

Now, you can see the kinds of questions the contest expects the ML to answer, just to take an example:

"Effectiveness of movement control strategies to prevent secondary transmission in health care and community settings"

All I can say is that the contest results, on the whole, were completely underwhelming. You can check out the Contributions page to verify this for yourself. If the consequences of the failure weren't so potentially catastrophic, some might even call it a little comical. I mean, its not as if a pandemic comes around every few months, so we can all just wait for the computational power to catch up to solve these problems like the author suggests.

Also, I couldn't help but feel that nearly all participants were more interested in applying the latest and greatest ML advancement (Bert QA!), often with no regard to the problem which was being solved.

I wish I could tell you I have some special insight into a better way to solve it, given that there is a friggin pandemic going on, and we could all very well do with some real friggin answers! I don't have any such special insight at all. All I found out was that everyone was so obsessed with using the latest and greatest ML techniques, that there was practically no first principles thinking. At the end, everyone just sort of got too drained and gave up, which is reflected by a single participant winning pretty much the entire second round of 7-8 task prizes by the virtue of being the last man standing :-)

I have realized two things.

1) ML, at least when it comes to understanding text, is really overhyped

2) Nearly everyone who works in ML research is probably overpaid by a factor of 100 (just pulling some number out of my you know what), given that the results they have actually produced have fallen so short precisely when they were so desperately needed

Re: The Bitter Lesson (2019)

#80

Post by coldtea »

>At the time, this was looked upon with dismay by the majority of computer-chess researchers who had pursued methods that leveraged human understanding of the special structure of chess.

This seems problematic as a concept in itself.

Sure human players have a "human understanding of the special structure of chess". But what makes them play could be an equally "deep search" and fuzzy computations done in the brain that and not some conscious step by step reasoning. Or rather, their "conscious step by step reasoning" to my opinion probably relies on tops on subconscious deep search in the brain for pruning the possible moves, etc.

I don't think anybody plays chess at any great level merely by making conscious step by step decisions.

Similar to how when we want to catch a ball thrown at us, we do some thinking like "they threw it to our right, so we better move right" but we also have tons of subconscious calculations of the trajectory (nobody sits and explicitly calculates the parabolic formula when they're thrown a baseball).

Post reply on HN