Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

newsweek.com

Post reply on HN
Print view View original

61–70 of 147 posts

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#61

Post by gh0stcat »

SirensOfTitan wrote: post #38
ObnoxiousProxy wrote: post #28

Earlier quoted context omitted.

this statement is just patently wrong, and even those are still significant value. LLMs have been significantly impacting software engineering and software prototyping.

Do you have any evidence to suggest that LLMs have increased productivity in software? And if so what is the effect size? I don’t really see any increase in the quality, velocity, creativity of software I’m either using or working on, but I have seen a ton of candidates with real experience flunk out on interviews because they forget the basics and blame it on LLMs. I’ve honestly never seen candidates struggle with s…

Yeah, even though people say they are shipping faster, the software my work pays for is not seemingly getting better at any faster rate.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#62

Post by logicchains »

SirensOfTitan wrote: post #38
ObnoxiousProxy wrote: post #28

Earlier quoted context omitted.

this statement is just patently wrong, and even those are still significant value. LLMs have been significantly impacting software engineering and software prototyping.

Do you have any evidence to suggest that LLMs have increased productivity in software? And if so what is the effect size? I don’t really see any increase in the quality, velocity, creativity of software I’m either using or working on, but I have seen a ton of candidates with real experience flunk out on interviews because they forget the basics and blame it on LLMs. I’ve honestly never seen candidates struggle with s…

It's hard to see where you're coming from. With o1 Pro or Gemini 2.5 Pro I can give a detailed text specification of a module I want to write to it and it'll write code to do exactly that, with fewer errors than I'd make on a first try. It'll also generate unit tests for it. Writing a text specification then reviewing some code for me is way, way faster than writing all that same code by scratch.

Are you working on code that's heavily coupled to other code, such that you rarely get to write independent logic components from scratch, and IP restrictions prevent you from handing large chunks of existing code to the LLM?

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#63

Post by throw310822 »

guhidalg wrote: post #32
GMoromisato wrote: post #18

I remember reading Douglas Hofstadter's Fluid Concepts and Creative Analogies [ https://en.wikipedia.org/wiki/Fluid_Concepts_and_Creative_An... ] He wrote about Copycat, a program for understanding analogies ("abc is to 123 as cba is to ???"). The program worked at the symbolic level, in the sense that it hard-coded a network of relationships between words and characters. I wonder how close he was to "inventing" an L…

I wouldn't call pattern matching intelligence, I would call it something closer to "trainability" or "educatable" but not intelligence. You can train a person to do a task without understanding why they have to do it like that, but when confronted with a new never-before-seen situation they have to understand the physical laws of the universe to find a solution. Ask ChatGPT to answer something that no one on the inte…

Pattern matching leads to compression- once you identified a pattern you can compress the original information by some amount by replacing it with the identified pattern. Patterns are symbols of the information that was there originally; so manipulating patterns is the same as manipulating symbols. Compressing information by finding hidden connections, then operating on abstract representations of the original information, reorganising this information according to other patterns... this sounds a lot like intelligence.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#65

Post by mordymoop »

gcr wrote: post #25
antirez wrote: post #23

As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…

Why is changing one’s mind when confronted with new evidence a negative signifier of reputation for you?

“Changing your mind” doesn’t really look like what LeCun is doing.

If your model of reality makes good predictions and mine makes bad ones, and I want a more accurate model of reality, then I really shouldn’t just make small provisional and incremental concessions gerrymandered around whatever the latest piece of evidence is. After a few repeated instances, I should probably just say “oops, looks like my model is wrong” and adopt yours.

This seems to be a chronic problem with AI skeptics of various sorts. They clearly tell us that their grand model indicates that such-and-such a quality is absolutely required for AI to achieve some particular thing. Then LLMs achieve that thing without having that quality. Then they say something vague about how maybe LLMs have that quality after all, somehow. (They are always shockingly incurious about explaining this part. You would think this would be important to them to understand, as they tend to call themselves “scientists”.)

They never take the step of admitting that maybe they’re completely wrong about intelligence, or that they’re completely wrong about LLMs.

Here’s one way of looking at it: if they had really changed their mind, then they would stop being consistently wrong.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#66

Post by cplat »

antirez wrote: post #41
jxjnskkzxxhx wrote: post #34

Earlier quoted context omitted.

I don't see the contradiction between "stochastic parrot" and "strong summarisation abilities". Where I'm skeptical of LLM skepticism is that people use the term "stochastic parrot" disparagingly, as if they're not impressed. LLMs are stochastic parrots in the sense that they probabilistically guess sequences of things, but isn't it interesting how far that takes you already? I'd never have guessed. Fundamentally I q…

LLMs learn from examples where the logits are not probabilities, but how a given sentence continues (only one token is set to 1). So they don't learn probabilities, they learn how to continue the sentence with a given token. We apply softmax at the logits for mathematical reasons, and it is natural/simpler to think in terms of probabilities, but that's not what happens, nor the neural networks they are composed of is…

I don't understand. Deterministic and stochastic have very specific meanings. The statement: "To continue my reply I could say this word, more than the others, or maybe that one, a bit less, ..." sounds very much like a probability distribution.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#67

Post by GMoromisato »

throw310822 wrote: post #63
guhidalg wrote: post #32

Earlier quoted context omitted.

I wouldn't call pattern matching intelligence, I would call it something closer to "trainability" or "educatable" but not intelligence. You can train a person to do a task without understanding why they have to do it like that, but when confronted with a new never-before-seen situation they have to understand the physical laws of the universe to find a solution. Ask ChatGPT to answer something that no one on the inte…

Pattern matching leads to compression- once you identified a pattern you can compress the original information by some amount by replacing it with the identified pattern. Patterns are symbols of the information that was there originally; so manipulating patterns is the same as manipulating symbols. Compressing information by finding hidden connections, then operating on abstract representations of the original inform…

Exactly! And once you compress a pattern, it can became a piece of a larger pattern.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#68

Post by strogonoff »

GeorgeTirebiter wrote: post #24
GMoromisato wrote: post #18

I remember reading Douglas Hofstadter's Fluid Concepts and Creative Analogies [ https://en.wikipedia.org/wiki/Fluid_Concepts_and_Creative_An... ] He wrote about Copycat, a program for understanding analogies ("abc is to 123 as cba is to ???"). The program worked at the symbolic level, in the sense that it hard-coded a network of relationships between words and characters. I wonder how close he was to "inventing" an L…

What is Dark Matter? How to eradicate cancer? How to have world peace? I don't quite see how pattern-matching, alone, can solve questions like these.

Pattern-matching can produce useful answers within the confines of a well-defined system. However, the hypothetical all-encompassing system for such a solver to produce hypothetical objective ground truth about an arbitrary question is not something we have—such a system would be one which we ourselves are part of and hence unavailable to us (cf. the incompleteness conundrum, map vs. territory, and so forth).

Your unsolved problems would likely involve the extremes of maps that you currently think in terms of. Maps become less useful as you get closer to undefined extreme conditions within them (a famous one is us humans ourselves, and why so many unsolved challenges to various degrees of obviousness concern our psyche and physiology—world peace, cancer, and so on), and I assume useful pattern matching is similarly less effective. Data to pattern-match against is collected and classified according to a preexisting model; if the model is wrong (which it is), the data may lead to spurious matches with wrong or nonsensical answers. Furthermore, if the answer has to be in terms of a new system, another fallible map hitherto unfamiliar to human mind, pattern-matching based on preexisting products of that very mind is unlikely to produce one.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#69

Post by fragmede »

jxjnskkzxxhx wrote: post #34
antirez wrote: post #26

Earlier quoted context omitted.

Because there were plenty of evidences that the statements were either not correct or not based on enough information, at the time they were made. And to be wrong because of personal biases, and then don't clearly state you were wrong when new evidenced appeared, is not a trait of a good scientist. For instance: the strong summarization abilities where already something that, alone, without any further information, w…

I don't see the contradiction between "stochastic parrot" and "strong summarisation abilities". Where I'm skeptical of LLM skepticism is that people use the term "stochastic parrot" disparagingly, as if they're not impressed. LLMs are stochastic parrots in the sense that they probabilistically guess sequences of things, but isn't it interesting how far that takes you already? I'd never have guessed. Fundamentally I q…

There are some that would describe LLMs as next word predictors, akin to having a bag of magnetic words, where you put your hand in, rummage around, and just pick a next word and put it on the fridge and eventually form sentences. It's "just" predicting the next word, so as an analogy as to how they work, that seems reasonable. The thing is, when that bag consists of a dozen bags-in-bags, like Russian nesting dolls, and the "bag" has a hundred million words in it, the analogy stops being a useful description. It's like describing humans as multicellular organisms. It's an accurate description of what a human is, but somewhere between a simple hydra with 100,000 cells and a human with 3 trillion cells, intelligence arises. Describing humans as merely multicellular organisms and using hydra as your point of reference isn't going to get you very far.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#70

Post by marliechiller »

aprilthird2021 wrote: post #46
GMoromisato wrote: post #18

I remember reading Douglas Hofstadter's Fluid Concepts and Creative Analogies [ https://en.wikipedia.org/wiki/Fluid_Concepts_and_Creative_An... ] He wrote about Copycat, a program for understanding analogies ("abc is to 123 as cba is to ???"). The program worked at the symbolic level, in the sense that it hard-coded a network of relationships between words and characters. I wonder how close he was to "inventing" an L…

> In fact, pattern-matching is all there is: That's a bear, run away; I'm in a restaurant, I need to order; this is like a binary tree, I can solve it recursively. This is not correct. It does not explain creativity at all. It cannot solely be based on pattern matching. I'm not saying no AI is creative, but this logic does not explain creativity

Is creativity not just the application of a pattern in an adjacent space?
Post reply on HN