Machine Learning: The Great Stagnation
marksaroufim.substack.com
Machine Learning: The Great Stagnation
1–10 of 227 posts
Re: Machine Learning: The Great Stagnation
#2Winter is coming, again.
Re: Machine Learning: The Great Stagnation
#3> It’s important to avoid becoming Gary Marcus and criticize existing technique that work without proposing something else that works even better.
Hilarious. Has Gary Marcus actually done anything, in practical terms, like actual code or something, that outperforms the DL approaches he attacks so viciously?
Re: Machine Learning: The Great Stagnation
#4> Academics sacrifice material opportunity costs in exchange for intellectual freedom.
Most academics I’ve come across only think they’re doing this. My perception is they are too insecure about their self-worth to pursue material opportunities.
I admit, the number of academic types I know is not vast so maybe it’s too small a subset to make any judgments
Re: Machine Learning: The Great Stagnation
#5I see a lot of parallels between ML/DL and cryptocurrency research.
Re: Machine Learning: The Great Stagnation
#6> It’s important to avoid becoming Gary Marcus and criticize existing technique that work without proposing something else that works even better. Hilarious. Has Gary Marcus actually done anything, in practical terms, like actual code or something, that outperforms the DL approaches he attacks so viciously?
I keep coming across versions of this comment.
Why does it matter? It’s not like the validity or invalidity of Gary’s criticisms would be different if he had done other totally separate ML research.
Re: Machine Learning: The Great Stagnation
#7Winter is coming, again.
Yeah, if I got a penny everytime I heard that one...
Re: Machine Learning: The Great Stagnation
#8Re: Machine Learning: The Great Stagnation
#9There's a nice talk by Yann LeCun where he goes on to explain really well why deep learning has such fast progress. [0]
He goes on to explain how theory always comes later.
I thought that information theory came before practice but turns out it also came after. (there was a bunch of heuristics for sending messages with teletypes)
A nice example is Roman technological advances in architecture and materials that predate the use of geometry or mathematics completely. All advancements a result of tinkering and heuristics.