Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“The Bitter Lesson” is wrong. Well sort of

assaf-pinhasi.medium.com

Post reply on HN
Print view View original

31–35 of 35 posts

Re: “The Bitter Lesson” is wrong. Well sort of

#31

Post by rhet0rica »

wasabi991011 wrote: post #28
suddenlybananas wrote: post #25

Earlier quoted context omitted.

Expert systems aren't a machine learning approach even if they're an AI approach.

Yes, but the Bitter Lesson is about AI, not ML.

Exactly. The expert system era was the first victim of the Bitter Lesson, as it was blown away when backpropagation was figured out at the end of the eighties.

An author familiar with the history of AI would have mentioned this instead of glossing it over as "not a learning model"—dismissing a problem-solving technique because it doesn't use regression serves no constructive purpose.

Re: “The Bitter Lesson” is wrong. Well sort of

#32

Post by littlestymaar »

rfv6723 wrote: post #26
littlestymaar wrote: post #16

The Leela Chess Zero vs Stockfish case also offers an interesting perspective on the bitter lesson. Here's my (maybe a bit loose) recollection of what happened: Step 1- Stockfish was the typical human-knowledge AI, with tons of actual chess knowledge injected in the process of building an efficient chess engine. Step 2. Then came Leela Chess Zero, with its Alpha Zero-inspired training, a chess engine trained fully wi…

Stockfish has got rid of old handwritten evaluation now. https://github.com/official-stockfish/Stockfish/pull/4674 Its evaluation now purely relies on NNUE neural network. So it's an good exmaple of the better lesson. More compute evently won against handwritten evaluation. Stockfish developers thought old evaluation would help neural network so they kept the code for a few years, then it turned out that NNUE neural…

AFAIK this is only a part of it, it still has its opening library, as well as the end-game one (IIRC chess is a solved game for up to seven remaining pieces on the board).

Also, the PR you link says the removed code in fact had a performance impact, just too low to justify its code size 25% of all Stockfish).

Re: “The Bitter Lesson” is wrong. Well sort of

#33

Post by layer8 »

ta8645 wrote: post #30
samrus wrote: post #20

Earlier quoted context omitted.

Bit fantastical. We are a general intelligence and we dont understand ourselves

Indeed. But the premise of the objection, was that it is understandable, and a shame that we're not putting such understanding before implementing these systems. If you're right, and it's essentially impossible to understand (and we still want to advance these technologies) we will have to do so in some degree of ignorance anyway.

It doesn’t have to be impossible to understand for (hypothetical) AGI having as much difficulty in understanding it as we do.

Re: “The Bitter Lesson” is wrong. Well sort of

#34

Post by ta8645 »

layer8 wrote: post #33
ta8645 wrote: post #30

Earlier quoted context omitted.

Indeed. But the premise of the objection, was that it is understandable, and a shame that we're not putting such understanding before implementing these systems. If you're right, and it's essentially impossible to understand (and we still want to advance these technologies) we will have to do so in some degree of ignorance anyway.

It doesn’t have to be impossible to understand for (hypothetical) AGI having as much difficulty in understanding it as we do.

I must not be communicating very well, because everyone is arguing with me about points i'm not trying to make. Sorry for that.
Post reply on HN