outside of text generation and search, LLMs have not delivered any significant value
Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
21–30 of 147 posts
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#22> Returning to the topic of the limitations of LLMs, LeCun explains, "An LLM produces one token after another. It goes through a fixed amount of computation to produce a token, and that's clearly System 1—it's reactive, right? There's no reasoning," a reference to Daniel Kahneman's influential framework that distinguishes between the human brain's fast, intuitive method of thinking (System 1) and the method of slower…
Re the "medical imaging" reference, many of those are built on top of one famous study recording movement before conscious realization that isn't as clear-cut as it entered popular knowledge as: https://www.theatlantic.com/health/archive/2019/09/free-will... I know there are other examples, and I'm not attacking your post; mainly it's a great opportunity to link this IMHO interesting article that interacts with many…
> IMHO interesting article that interacts with many debates on HN.
It's paywalled
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#23As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between the words of a text and the gist of the content, still he was saying that at the time of GPT3.5 I believe. Then he adjusted this vision when talking with Hinton publicly, saying "I don't deny there is more than just probabilistic thing...". He started saying: not longer just simply probabilistic but they can only regurgitate things they saw in the training set, often explicitly telling people that novel questions could NEVER solved by LLMs, with examples of prompts failing at the time he was saying that and so forth. Now reasoning models can solve problems they never saw, and o3 did huge progresses on ARC, so he adjusted again: for AGI we will need more. And so forth.
So at this point it does not matter what you believe about LLMs: in general, to trust LeCun words is not a good idea. Add to this that LeCun is directing an AI lab that as the same point has the following huge issues:
1. Weakest ever LLM among the big labs with similar resources (and smaller resources: DeepSeek).
2. They say they are focusing on open source models, but the license is among the less open than the available open weight models.
3. LLMs and in general all the new AI wave puts CNNs, a field where LeCun worked (but that didn't started himself) a lot more in perspective, and now it's just a chapter in a book that is composed mostly of other techniques.
Btw, other researchers that were in the LeCun side, changed side recently, saying that now "is different" because of CoT, that is the symbolic reasoning they were blabling before. But CoT is stil regressive next token without any architectural change, so, no, they were wrong, too.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#24I remember reading Douglas Hofstadter's Fluid Concepts and Creative Analogies [ https://en.wikipedia.org/wiki/Fluid_Concepts_and_Creative_An... ] He wrote about Copycat, a program for understanding analogies ("abc is to 123 as cba is to ???"). The program worked at the symbolic level, in the sense that it hard-coded a network of relationships between words and characters. I wonder how close he was to "inventing" an L…
What is Dark Matter? How to eradicate cancer? How to have world peace? I don't quite see how pattern-matching, alone, can solve questions like these.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#25As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…
Why is changing one’s mind when confronted with new evidence a negative signifier of reputation for you?
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#26As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…
Why is changing one’s mind when confronted with new evidence a negative signifier of reputation for you?
Because there were plenty of evidences that the statements were either not correct or not based on enough information, at the time they were made. And to be wrong because of personal biases, and then don't clearly state you were wrong when new evidenced appeared, is not a trait of a good scientist. For instance: the strong summarization abilities where already something that, alone, without any further information, were enough to seriously doubt about the stochastic parrot mental model.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#27outside of text generation and search, LLMs have not delivered any significant value
Text generation and search are the drivers for some trillions of dollars worth of economic activity around the world.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#28outside of text generation and search, LLMs have not delivered any significant value
this statement is just patently wrong, and even those are still significant value. LLMs have been significantly impacting software engineering and software prototyping.
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#29outside of text generation and search, LLMs have not delivered any significant value
Would you mind elaborating on how LLMs have not delivered any significant value outside of text generation and search?
Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete
#30Earlier quoted context omitted.
Why is changing one’s mind when confronted with new evidence a negative signifier of reputation for you?
Because there were plenty of evidences that the statements were either not correct or not based on enough information, at the time they were made. And to be wrong because of personal biases, and then don't clearly state you were wrong when new evidenced appeared, is not a trait of a good scientist. For instance: the strong summarization abilities where already something that, alone, without any further information, w…
Here's a fun example of that kind of "I've updated my statements but not assessed any of my underlying lack of understanding" - it's a bad look on any kind of scientist.