The Bitter Lesson (2019)
incompleteideas.net
The Bitter Lesson (2019)
1–10 of 87 posts
Re: The Bitter Lesson (2019)
#2It’s funny when you’ve been thinking for months about how speech recognition could really benefit from integrating models of the human vocal tract…
and then you read this
Re: The Bitter Lesson (2019)
#3It’s funny when you’ve been thinking for months about how speech recognition could really benefit from integrating models of the human vocal tract… and then you read this
"Every time I fire an anatomist and hire a TPU pod, my WER halves."
Re: The Bitter Lesson (2019)
#4Got to believe, this is like heroin. Its a win until it isn't. Then where will AI researchers be? No progress for 20 (50?) years because the temptation to not understand but to just build performant engineering solutions, was so strong.
In fact, is the researcher supposed to be building the most performant solution? This article seems alarmingly misinformed. To understand 'artificial intelligence' isn't a race to VC money.
Re: The Bitter Lesson (2019)
#5I have read this before and broadly agree with the point—it’s no use trying to curate expertise into AI. But I don’t think modeling p(y|x) or it’s friend p(y, x) is the end we’re looking for either. But, it’s unreasonably effective, so we keep doing it. (I don’t have an answer or an alternative; causality appeals to my intuition, but it’s really clunky and has seemingly not paid off.)
Re: The Bitter Lesson (2019)
#6TL;DR: AI needs a hand up, not a handout.
"We want AI agents that can discover like we can, not which contain what we have discovered."
I was internally protesting all the way through the note, until I got to that penultimate sentence.
Re: The Bitter Lesson (2019)
#7This page seems to be down. Is there a mirror?
Re: The Bitter Lesson (2019)
#8This lesson can be applied to synthetic biology right now, though it is still in its infant stages.
At least a few of the original synthetic biologists are a bit disappointed in the rise of high-throughput testing for everything, instead of "robust engineering". Perhaps what allows us to understand life isn't just more science, but more "biotech computation".
Re: The Bitter Lesson (2019)
#9I guess it depends on what you trying to do. I had a computer vision problem where I was like, hell yeah, let's machine learn the hell out of this. 2 months later, and the results were just not precise enough. It took me 2 more months, and now I am solving the task easily on an iPhone via Apple Metal in milliseconds with a hand-crafted optimisation approach ...
Re: The Bitter Lesson (2019)
#10TL;DR: AI needs a hand up, not a handout. "We want AI agents that can discover like we can, not which contain what we have discovered." I was internally protesting all the way through the note, until I got to that penultimate sentence.
Yeah, it takes a careful, charitable reading to not interpret it as "don't bother with understanding or finding new methods, just throw more FLOPS at it".