Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Bitter Lesson (2019) [pdf]

cs.utexas.edu

Post reply on HN
Print view View original

51–60 of 62 posts

Re: The Bitter Lesson (2019) [pdf]

#51

Post by cubefox »

gradascent wrote: post #47
cubefox wrote: post #33

Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…

If I remember correctly, the algorithm in Alpha Go used a combination of reinforcement learning and searching the space of moves and possible outcomes to determine the next move. So it does indeed use both search and learning. Regardless, the author's point is that computation is a better way of finding and exploiting patterns/strategies than our own intuitions. The distinction between search and learning is not the…

In hindsight computation wasn't the important thing though. A lot of things require a lot of computation that aren't intelligent or don't scale well, like Deep Blue. The important breakthrough in AI was learning ("machine learning").

Re: The Bitter Lesson (2019) [pdf]

#52

Post by nextworddev »

The money quote (basically says why deep learning will keep on dominating): "One thing that should be learned from the bitter lesson is the great power of general purpose methods, of methods that continue to scale with increased computation even as the available computation becomes very great. The two methods that seem to scale arbitrarily in this way are search and learning."

Re: The Bitter Lesson (2019) [pdf]

#53

Post by milesvp »

gradascent wrote: post #47
cubefox wrote: post #33

Sutton cites people who were not convinced by the success of Deep Blue in chess: > They said that ``brute force" search may have won this time, but it was not a general strategy, and anyway it was not how people played chess. But it is now clear that these people were completely right! The Deep Blue approach (apparently mostly brute force search) didn't scale. E.g. Go wasn't solved this way, and the AlphaGo approach…

If I remember correctly, the algorithm in Alpha Go used a combination of reinforcement learning and searching the space of moves and possible outcomes to determine the next move. So it does indeed use both search and learning. Regardless, the author's point is that computation is a better way of finding and exploiting patterns/strategies than our own intuitions. The distinction between search and learning is not the…

There was an important step prior to alpha go. At the time the combinatorics were in favor of Go. But someone had the bright idea to do a probabalistic search of the space. The key idea was to play a ton of random games and rate each position based what percentage that spot was included in winning games. This blew away all other go ai at the time. Sadly this was about the time I stopped having time to follow the space, so I’m not sure how this idea was further incorporated in Go AI. But it was truly a revolutionary idea at the time

Re: The Bitter Lesson (2019) [pdf]

#54

Post by tysam_and »

Hi, I say this so much that it is almost a revolving PSA at this point, but here we go again:

Big reminder that the Bitter Lesson is _not_ saying "just scale your methods and they work". What the Bitter Lesson _is_, however, is "work on methods that scale". There is a _huge_ distinction between the two, in my opinion.

Effectively, if I can add my layer of (re?)interpretation on it, it's saying that specialized boutique, hand-designed solutions don't play very well in the long-term in an arena with Moore's Law and money. But, what it's not saying, is that just making an algorithm bigger is the solution. This is how I see most people missing its interpretation.

For an algorithm to be effective, it needs 3 things in my opinion: 1. It needs to scale (measurable by some factor) 2. Because we have 1, it needs to have an implementation with extraordinarily rapid iteration time 3. If we have 2, we need an implementation that is extraordinarily lightweight (enough to run on consumer machines)

These three factors together, in my personal estimation, unlock algorithmic research progress in an area.

An ancillary fourth rule that really drives progress IMO is competition, formalized or otherwise, that is 1. open, 2. well-known, 3. incentivized and 4. accessible.

This is my personal opinion of course, and bound to be flawed in some way, but -- from personal experience, at least -- when I've seen this field (or research fields in general) align with this kind of method of research is where the speed of algorithmic research has absolutely exploded. :) <3

Re: The Bitter Lesson (2019) [pdf]

#55

Post by imtringued »

nologic01 wrote: post #43
Xcelerate wrote: post #35

Earlier quoted context omitted.

> This (popular) opinion makes no sense. How are you going to improve something you dont understand? How does that opinion not make sense? There are numerous things humans have invented for which we have little understanding of how they work: medical drugs, anesthesia, certain quantum phenomena utilized in semiconductors, etc. I would argue that for current state-of-the-art LLMs, the implementation is likewise ahead…

"Understanding" does not need internalizing a process or algorithm at low level. We are not stochastic parrots. It is simply possesing sufficient insight so as to be able to explore nearby designs, formulate and test hypotheses, narrow the search space etc. There is a strange emerging AI cult that is also in force here in HN that seems to believe these algorithms have evolved themselves or were some random trial and…

For decades ML researchers were walking on eggshells, thinking that they are going to run into overfitting and bias/variance tradeoffs, adding regularisation, worrying about local maxima, etc. Then one day a certain company kept making their models bigger without any concern and suddenly everyone thinks that there are unlimited returns to scale, when in reality you merely bought yourself a one time victory.

Re: The Bitter Lesson (2019) [pdf]

#56

Post by tysam_and »

PartiallyTyped wrote: post #15
PoignardAzur wrote: post #11

Earlier quoted context omitted.

Ehh, I'm more on Sutton's side so far. Brook's post goes over the classics (Moore's law is ending, curating a dataset requires human intervention, etc) and posits that making a huge model won't be a competitive strategy for long because it gets to expensive to train and use. It's a bit early to tell, but so far that hasn't materialized. OpenAI got state-of-the-art results with GPT-4, AFAIK by sticking for very-super-…

Going really big and figuring out how to go small is also working; see Lottery Tickets, ensembles, distillation, and so on.

As someone who holds a speed WR with a very tiny model (If you do not have solid scaling, your link between micro methods predicting mega methods is broken. Hence, Sutton's bitter law forms the foundation for a few other lemmas that I think underpin what makes really effective research (which is iteration time, and how we effectively reduce it as much as possible and make it as accessible as possible -- which thankfully for ML algorithms seems to go hand in hand! <3 :')))) )

Re: The Bitter Lesson (2019) [pdf]

#57

Post by laughingman2 »

cubefox wrote: post #51
gradascent wrote: post #47

Earlier quoted context omitted.

If I remember correctly, the algorithm in Alpha Go used a combination of reinforcement learning and searching the space of moves and possible outcomes to determine the next move. So it does indeed use both search and learning. Regardless, the author's point is that computation is a better way of finding and exploiting patterns/strategies than our own intuitions. The distinction between search and learning is not the…

In hindsight computation wasn't the important thing though. A lot of things require a lot of computation that aren't intelligent or don't scale well, like Deep Blue. The important breakthrough in AI was learning ("machine learning").

Search/Reasoning/Inference time compute, however you phrase it is still essential. You need search to improve upon learning to work in novel situations.

Re: The Bitter Lesson (2019) [pdf]

#58

Post by cubefox »

laughingman2 wrote: post #57
cubefox wrote: post #51

Earlier quoted context omitted.

In hindsight computation wasn't the important thing though. A lot of things require a lot of computation that aren't intelligent or don't scale well, like Deep Blue. The important breakthrough in AI was learning ("machine learning").

Search/Reasoning/Inference time compute, however you phrase it is still essential. You need search to improve upon learning to work in novel situations.

Humans do very little "search", as we can see in games like Go. Which suggests it isn't essential for AI either.

Re: The Bitter Lesson (2019) [pdf]

#59

Post by the8472 »

reaperman wrote: post #40
the8472 wrote: post #32

Earlier quoted context omitted.

> GPT4 was basically trained on all text Well, there's multi-modal training. There's tons of untapped audiovisual data.

GPT4 is a multi-modal model. They haven't exposed a way to use the image embeddings, so consumers can't utilize it, but the model accepts image input and it was trained on images and text. Yes, there are other modals that can be incorporated, and reinforcement learning is still pretty nascent/very much unsolved.

The fact that it's 1) multi-modal and 2) we're approaching text exhaustion does not imply 3) that other modalities are exhausted

Re: The Bitter Lesson (2019) [pdf]

#60

Post by reaperman »

the8472 wrote: post #59
reaperman wrote: post #40

Earlier quoted context omitted.

GPT4 is a multi-modal model. They haven't exposed a way to use the image embeddings, so consumers can't utilize it, but the model accepts image input and it was trained on images and text. Yes, there are other modals that can be incorporated, and reinforcement learning is still pretty nascent/very much unsolved.

The fact that it's 1) multi-modal and 2) we're approaching text exhaustion does not imply 3) that other modalities are exhausted

Yeah I just like pointing out that GPT-4 is multimodal because most people don't seem to realize this, as they never read the GPT-4 papers.
Post reply on HN