41–50 of 62 posts
this thread on a "bitter lesson 2.0" relating to robotics from earlier this year is interesting https://twitter.com/hausman_k/status/1613544836266885120
Earlier quoted context omitted.
The essence of the bitter lesson is that the less inductive bias we try to bake in, the better performing they tend to be. Transformers are preferred to ConvNets these days in Computer Vision despite the latter having all sorts of vision based inductive biases. GPT-4 has not been trained on all text lol
1. Partially that just moves where we must search for implicitly included symmetries. Word embedding most famously create structure that allows to define shift operations in the embedding space “doctor + female = nurse” might be the most infamous. By evolution only those word embedding so that yield good results are used. Again just because we did not put the structure there, but discovered it by trial and error does…
2. Assuming the rumor is fact, GPT was trained for multiple epochs and books are a small percent of what trains LLMs lol.
Earlier quoted context omitted.
This (popular) opinion makes no sense. How are you going to improve something you dont understand? Throwing ramdom software pieces to see what sticks? The recent progress was possible because people understood intuitively enough of the limitations of earlier models to think and invent a remedy.
> This (popular) opinion makes no sense. How are you going to improve something you dont understand? How does that opinion not make sense? There are numerous things humans have invented for which we have little understanding of how they work: medical drugs, anesthesia, certain quantum phenomena utilized in semiconductors, etc. I would argue that for current state-of-the-art LLMs, the implementation is likewise ahead…
There is a strange emerging AI cult that is also in force here in HN that seems to believe these algorithms have evolved themselves or were some random trial and error. Ergo, they can keep evolving and the researchers dont need to understand a thing about how they work.
Serendipity in combining ideas that prove effective plays a role but within a fairly well defined conceptual sandbox. But progress with AI is more or less conditional on people having sufficient understanding to coax algorithmic structures in the desired direction.
This is the original page, I’d link to this: http://www.incompleteideas.net/IncIdeas/BitterLesson.html u/dang, swap links if you see this?
Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…
Pretrained dense GPTs don't search in any conventional way. However, when these GPTs are fine-tuned using RL, an aspect of search is reintroduced. One the most interesting directions labs are pursuing to improve on today's best LLMs is to fine-tune them with search (RL) after a learning-based (next-token prediction) pre-training.
Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…
I think they're currently solving it for eight pieces.
So it's learning but then brute-force for the end game.
Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…
Regardless, the author's point is that computation is a better way of finding and exploiting patterns/strategies than our own intuitions. The distinction between search and learning is not the important one here.
Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…
We train deep learning models with SGD, which is in fact a form of search -- in a high-dimensional parameter space. The purpose of SGD is to find parameter values that minimize a training loss.
Earlier quoted context omitted.
>> Go has been described as the drosophila of AI. John McCarthy described chess as "the drosophila of AI": http://jmc.stanford.edu/articles/drosophila/drosophila.pdf Who was it that described Go as the drosophila of AI?
It was John McCarthy, in the very paper that you cite! See section 7: As a fourth Drosophila I would like to mention the research on Computer Go.
Thanks for the correction.
Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…
SGD is a method of learning from examples (mapping X -> y). RL is a method of search (maximizing reward signal in a potentially open-ended environment, based on feedback on the actions that a model takes). Go was solved by combining learning and search (e.g. narrowing the search space with learned, educated guesses). Pretrained dense GPTs don't search in any conventional way. However, when these GPTs are fine-tuned u…