Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

Post by gwd » Sat, Apr 05, 2025, 5:59 PM UTC

falcor84 wrote: post #36

asdev wrote: post #21

outside of text generation and search, LLMs have not delivered any significant value

I personally have greatly benefitted from LLM's helping me reason about problems and make progress on many diverse issues across professional, recreational and mental health difficulties. I think that asking whether it's just "text generation and search" rather than something that transcends it is as meaningful as asking whether an airplane really "flies" or just "applies thrust and generates lift".

Post by QuantumGood » Sat, Apr 05, 2025, 6:01 PM UTC

gcr wrote: post #25

antirez wrote: post #23

As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…

Why is changing one’s mind when confronted with new evidence a negative signifier of reputation for you?

Post by fragmede » Sat, Apr 05, 2025, 6:05 PM UTC

SJC_Hacker wrote: post #59

danielmarkbruce wrote: post #57

Earlier quoted context omitted.

If you need basically rock solid evidence of X before you stop saying "this thing cannot do X", then you shouldn't be running a forward looking lab. There are only so many directions you can take, only so many resources at your disposal. Your intuition has to be really freakishly good to be running such a lab. He's done a lot of amazing work, but his stance on LLMs seems continuously off the mark.

The list of great minds who thought that "new fangled thing is nonsense" and later turned out to be horribly wrong is quite long and distinguished

Post by antirez » Sat, Apr 05, 2025, 6:11 PM UTC

cplat wrote: post #66

antirez wrote: post #41

Earlier quoted context omitted.

LLMs learn from examples where the logits are not probabilities, but how a given sentence continues (only one token is set to 1). So they don't learn probabilities, they learn how to continue the sentence with a given token. We apply softmax at the logits for mathematical reasons, and it is natural/simpler to think in terms of probabilities, but that's not what happens, nor the neural networks they are composed of is…

I don't understand. Deterministic and stochastic have very specific meanings. The statement: "To continue my reply I could say this word, more than the others, or maybe that one, a bit less, ..." sounds very much like a probability distribution.

Post by fragmede » Sat, Apr 05, 2025, 6:13 PM UTC

belter wrote: post #72

antirez wrote: post #23

As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…

But have we established that LLMs dont just interpolate and they can create? Are we able to prove it with output that's 1) algorithmically novel (not just a recombination) 2) coherent, and 3) not explainable by training data coverage. No handwaving with scale...