Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

newsweek.com

Post reply on HN
Print view View original

131–140 of 147 posts

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#131

Post by kristianp »

mbesto wrote: post #89
antirez wrote: post #23

As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…

I wanna believe everything you say (because you generally are a credible person) but a few things don't add up: 1. Weakest ever LLM? This one is really making me scratch my head. For a period of time Llama was considered to THE best. Furthermore, it's the third most used on OpenRouter (in the past month): https://openrouter.ai/rankings?view=month 2. Ignoring DeepSeek for a moment, Llama 2 and 3 require a special lice…

I've found the llama 3 served by meta.ai to be quite weak for coding prompts, it gets confused by more complex tasks. Maybe its a smaller model? I agree it's weaker than others of its generation.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#132

Post by [deleted] »

wat10000 wrote: post #97
antirez wrote: post #23

As LLMs do things thought to be impossible before, LeCun adjusts his statements about LLMs, but at the same time his credibility goes lower and lower. He started saying that LLMs were just predicting words using a probabilistic model, like a better Markov Chain, basically. It was already pretty clear that this was not the case as even GPT3 could do summarization well enough, and there is no probabilistic link between…

LLMs literally are just predicting tokens with a probabilistic model. They’re incredibly complicated and sophisticated models, but they still are just incredibly complicated and sophisticated models for predicting tokens. It’s maybe unexpected that such a thing can do summarization, but it demonstrably can.

[deleted]

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#133

Post by nurettin »

danielmarkbruce wrote: post #130
nurettin wrote: post #119

Earlier quoted context omitted.

I'm going to wear the tinfoil hat: a firm is able to produce a sought-after behavior a few months later and throws people off. Is it more likely that the firm (worth billions at this point) is engineering these solutions into the model, or is it because of emergent neural network architectural magic? I'm not saying that they are being bad actors, just saying this is more probable in my mind than an LLM breakthrough.

It depends what you mean by "engineering these solutions into the model". Using better data leads to better models given the same architecture and training. Nothing wrong with it, it's hard work, it might be with as specific goal in mind. LLM "breakthroughs" aren't really a thing at this point. It's just one little thing after another.

Sure, I specifically pre-agreed to it not being ill will. What I mean is keeping tabs on the latest demand (newer benchmarks) and making sure their model delivers in some fashion. But it is mundane and they don't say that. And when a major number increases, people don't assume they just added more specific training data.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#134

Post by an_guy »

mdp2021 wrote: post #81
baumy wrote: post #27

Earlier quoted context omitted.

Text generation and search are the drivers for some trillions of dollars worth of economic activity around the world.

> trillions of dollars That is monetary value. The poster may have meant "delivery" value - which has been limited (and tainted with hype). > Text generation Which «text generation», apart from code generation (quite successful in some models), would amount to «trillions of dollars worth of economic activity» at the current stage? I cannot see it at the moment.

I cannot see any major use case apart from code generation and autocompletion. Maybe summarizing text and learning new things but that can be achieved by any search engine.

A good search engine could probably take over llm.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#135

Post by gugagore »

gibsonf1 wrote: post #124
airstrike wrote: post #113

Earlier quoted context omitted.

I'm ignorantly curious of what type of math will work in your view. Genuine question, I just want to be educated.

There is no type of math that can model conceptual reasoning. You can use conceptual reasoning, however, to do math.

I think I know what math is, though I'm not sure. Logical systems of axioms and inference rules?

But I'm even less sure what conceptual reasoning is.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#136

Post by gsf_emergency_2 »

csdvrx wrote: post #16
gibsonf1 wrote: post #6

Earlier quoted context omitted.

The error with that is that human reasoning is not mathematical. Math is just one of the many tools of reason.

Intransitive preferences is well known to experimental economists, but a hard pill to swallow for many, as it destroys a lot of algorithms (which depends on that) and require more robust tools like https://en.wikipedia.org/wiki/Paraconsistent_logic > just one of the many tools of reason. Read https://en.wikipedia.org/wiki/Preference_(economics)#Transit... then read https://pmc.ncbi.nlm.nih.gov/articles/PMC7058914/ an…

Thanks for the links, the "tradeoff" aspect of paraconsistent logic is interesting. I think one way to achieve consensus with your debate partner might be to consider that the language rep is "just" a nondeterministic decompression of "the facts". I'm primed to agree with you but

https://news.ycombinator.com/item?id=41892090

(It's very common, esp. with educationally traumatized Americans, e.g., to identify Math with "calculation"/"approved tools" and not "the concepts")

"No amount of calculation will model conceptual thinking" Sorry to come across as patronizing

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#138

Post by 082349872349872 »

gsf_emergency_2 wrote: post #136
csdvrx wrote: post #16

Earlier quoted context omitted.

Intransitive preferences is well known to experimental economists, but a hard pill to swallow for many, as it destroys a lot of algorithms (which depends on that) and require more robust tools like https://en.wikipedia.org/wiki/Paraconsistent_logic > just one of the many tools of reason. Read https://en.wikipedia.org/wiki/Preference_(economics)#Transit... then read https://pmc.ncbi.nlm.nih.gov/articles/PMC7058914/ an…

Thanks for the links, the "tradeoff" aspect of paraconsistent logic is interesting. I think one way to achieve consensus with your debate partner might be to consider that the language rep is "just" a nondeterministic decompression of "the facts". I'm primed to agree with you but https://news.ycombinator.com/item?id=41892090 (It's very common, esp. with educationally traumatized Americans, e.g., to identify Math with…

if conceptual thinking is manipulating abstract concepts after having been given concrete particulars, I'd say it relies heavily upon projection, which, as generalised "K" (from SKI), sounds awfully like calculation.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#139

Post by aerhardt »

neom wrote: post #83
DrBenCarson wrote: post #49

Earlier quoted context omitted.

It’s not true The idea that meaning is not impacted by language yet is somehow exclusively captured by language is just absolutely absurd Like saying X+Y=Z but changing X or Y won’t affect Z

Language is a symbolic system. From an absolute or spiritual standpoint, meaning transcends pure linguistic probabilities. Language itself emerges as a limited medium for the expression of consciousness and abstract thought. Indeed, to say meaning arises purely from language (as probability alone) or, to deny language influences meaning entirely are both overly simplistic extremes.

"When he to whom one speaks does not understand, and he who speaks himself does not understand, that is metaphysics." - Voltaire

Like I said in another comment, I can think of a dozen statistical and computational methods where if you give me a text and its synthesis I can find a strong probabilistic link between the two.

Re: Yann LeCun, Pioneer of AI, Thinks Today's LLM's Are Nearly Obsolete

#140

Post by klipt »

ksec wrote: post #90
klipt wrote: post #55

Earlier quoted context omitted.

> LLMs learn from examples where the logits are not probabilities, but how a given sentence continues (only one token is set to 1). But enough data implies probabilities. Consider 2 sentences: "For breakfast I had oats" "For breakfast I had eggs" Training on this data, how do you complete "For breakfast I had..."? There is no best deterministic answer. The best answer is a 50/50 probability distribution over "oats" a…

So it is still largely, probabilities pattern matching?

You can model the whole universe with probabilities!
Post reply on HN