Game over for pure LLMs. Even Rich Sutton has gotten off the bus
garymarcus.substack.com
Game over for pure LLMs. Even Rich Sutton has gotten off the bus
1–10 of 53 posts
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#2[deleted]
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#3Gary Marcus saying the same things Gary Marcus has always said.
It doesn’t matter what incredible things neural networks do, in his mind they’re always a dead end that will collapse any day now.
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#4> ... major thinker ...
Thesis: language isn't a great representation, basically.
I really should apply myself. Maybe I wouldn't work so hard, just shuck nonsense/pontificate.
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#5Gary Marcus has been taking victory laps on this since mid-2023, nothing to see here. Patently obvious to all that there will be additional innovations on top of LLMs such as test-time compute, which nonetheless are structured around LLMs and complementary
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#6Gary Marcus saying the same things Gary Marcus has always said. It doesn’t matter what incredible things neural networks do, in his mind they’re always a dead end that will collapse any day now.
There’s quite a gulf of difference between saying something is a dead end to full on general artificial intelligence, and saying it’s all dead and will collapse.
I have no idea if LLMs will be general AI, but they defo aren’t going anywhere
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#7Looking at the quoted tweet it is immediately obvious that these people have no clue about the current state of research. Yes they might have had some more or less relevant contributions to classical ML, but AI has taken off without (or rather despite) them and if history of AI has shown anything, it's that people like those are not the ones who will pave the way forward. In a field like this, there's no use to listen to people who still cling to their old ideas just because the current ideas don't seem "elegant" or "right" in their mind. The only thing you can trust is data and it proves we haven't peaked yet when it comes to LLMs.
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#8Anyone have links for these:
> Yann LeCun was first, fully coming around to his own, very similar critique of LLMs by end of 2022.
> The Nobel Laureate and Google DeepMind CEO Sir Demis Hssabis sees it now, too.
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#9[flagged]
Re: Game over for pure LLMs. Even Rich Sutton has gotten off the bus
#10Gary Marcus talks about AI in a dogmatic way.
Sutton... the patron saint of scaling...
Listen to people for the their ideas, not their label.
Regardless, Marcus is a bit late to comment on the bitter lesson. That is so 6 months ago lol