Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun to depart Meta and launch AI startup focused on 'world models'

nasdaq.com

Post reply on HN
Print view View original

651–660 of 680 posts

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#651

Post by tonii141 »

snapcaster wrote: post #429
tonii141 wrote: post #254

Earlier quoted context omitted.

I’m not demanding anything, I’m pointing out that performance tends to degrade as context scales, which follows from current LLM architectures as autoregressive models. In that sense, Yann was right.

Not sure if you're just someone who doesn't want to ever lose an argument or you're actually coping this hard

I just see a lot of people who’ve put money in the LLM basket and get scared by any reasonable comment about why LLMs aren’t almighty AGIs and may never be. Or maybe they are just dumb, idk.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#652

Post by ACCount37 »

tonii141 wrote: post #651
snapcaster wrote: post #429

Earlier quoted context omitted.

Not sure if you're just someone who doesn't want to ever lose an argument or you're actually coping this hard

I just see a lot of people who’ve put money in the LLM basket and get scared by any reasonable comment about why LLMs aren’t almighty AGIs and may never be. Or maybe they are just dumb, idk.

Even the bold take of "LLMs are literally AGI right now" is less of a detour from reality than "LLMs are NEVER going to hit AGI".

We've had LLMs for 5 years now, and billions were put into pushing them to the limits. We are yet to discover any fundamental limitations that would prevent them from going all the way to AGI. And every time someone pops up with "LLMs can never do X", it's followed up by an example of LLMs doing X.

Not that it stops the coping. There is no amount of evidence that can't be countered by increasing the copium intake.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#653

Post by killerstorm »

tonii141 wrote: post #102
NitpickLawyer wrote: post #39

Earlier quoted context omitted.

He also said other things about LLMs that turned out to be either wrong or easily bypassed with some glue. While I understand where he comes from, and that his stance is pure research-y theory driven, at the end of the day his positions were wrong. Previously, he very publicly and strongly said: a) LLMs can't do math. They trick us in poetry but that's subjective. They can't do objective math. b) they can't plan c) b…

a) Still true: vanilla LLMs can’t do math, they pattern-match unless you bolt on tools. b) Still true: next-token prediction isn’t planning. c) Still true: error accumulation is mitigated, not eliminated. Long-context quality still relies on retrieval, checks, and verifiers. Yann’s claims were about LLMs as LLMs. With tooling, you can work around limits, but the core point stands.

LeCun's argument was that a single erroneous token would derail further response.

This is, obviously, false: a reasoning model (or a non-reasoning one with a better prompt) can recognize error and choose a different path, the error will not be the part of an answer.

You're talking about a different problem: context rot. It's possible that an error would make performance worse. So what?

People can also get tired when they are solving a complex problem. People use various mitigations: e.g. it might help to start from a clean sheet. These mitigations might also apply to LLM: e.g. you can do MCTS (tree-of-thought) or just edit reasoning trace replacing the faulty part.

"LLMs are not absolutely perfect and require some algorithms on top thus we need a completely different approach" is a very weird way to make a conclusion.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#654

Post by barrkel »

fauigerzigerk wrote: post #425
pton_xd wrote: post #352

Earlier quoted context omitted.

> People may well have a fuzzy idea of how their credit card works, but how it really works is formally defined by financial institutions. > Our probabilistic, fuzzy concepts are often simply a misconception. How eg a credit card works today is defined by financial institutions. How it might work tomorrow is defined by politics, incentives, and human action. It's not clear how to model those with formal language. I t…

To some degree I think that our widely used formal languages may just be insufficient and could be improved to better describe change. But ultimately I agree with you that this entire societal process is just categorically different. It's simply not a description or definition of something, and therefore the question of how formal it can be doesn't really make sense. Formalisms are tools for a specific but limited pu…

I believe the formalisms can be constructed by something fuzzy. Humans are fuzzy; they create imperefect formalisms that work until they break, and then they're abandoned or adapted.

I don't see how LLMs are significantly different. I don't think the formalisms are an "other". I believe they could be tools, both leveraged and maintained by the LLM, in much the same way as most software engineers, when faced with a tricky problem that is amenable to brute force computation, will write up a quick script to answer it rather than try and work it out by hand.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#656

Post by HarHarVeryFunny »

astrange wrote: post #624
HarHarVeryFunny wrote: post #594

Earlier quoted context omitted.

There are many factors playing into "survival of the fittest", and energy conservation is only one. Animals build mental models to predict the world because this superpower of seeing into the future is critical to survival - predict where the water is in a drought, where the food is, and how to catch it, etc, etc. The animal learns as it encounters learning signals - prediction failure - which is the only way to do i…

It doesn't necessarily need to model the world to learn how to perform actions though. That was the topic of this old GOFAI research: https://aaai.org/papers/00268-aaai87-048-pengi-an-implementa... It instead works by "doing the thing that worked last time". As an example, you don't usually need to know what is in your garbage in order to take out the trash.

Right - I've no idea how LeCun thinks about it, but I don't see that an animal needs or would have any more of a "world model" than something like an LLM. I'm sure all the research into rats in mazes etc has something to say about their representations of location/etc, but given a goal of prediction it seems that all is needed is a combination of pattern recognition and sequence prediction - not an actual explicit "declarative" model.

It seems that things like place cells and grandmother cells are a part of the pattern recognition component, but recognizing landmarks and other predictive-relevant information doesn't mean we have a complete coherent model of the environments we experience - perhaps more likely a fragmented one of task-relevant memories. It seems like our subjective experience of driving is informative - we don't have a mental road map but rather familiarity with specific routes and landmarks. We know to turn right at the gas station, etc.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#657

Post by ta12653421 »

HarHarVeryFunny wrote: post #550
potatoproduct wrote: post #525

Earlier quoted context omitted.

Anthropic feels like a one trick pony as most users dont need or want anthropic products. However, I speak with a small subset of our most experienced engineers and they all love Claude Sonnet 4.5. Who knows if this lead will last.

Anthropic are mostly selling, and having most success, with business customers (incl. selling API access for Claude Code). This is the reason they haven't bothered to provide an image generator yet - because Chat users are not their focus.

Lately ClaudeAI switched over to ASCII art when doing explanations....

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#658

Post by fauigerzigerk »

barrkel wrote: post #654
fauigerzigerk wrote: post #425

Earlier quoted context omitted.

To some degree I think that our widely used formal languages may just be insufficient and could be improved to better describe change. But ultimately I agree with you that this entire societal process is just categorically different. It's simply not a description or definition of something, and therefore the question of how formal it can be doesn't really make sense. Formalisms are tools for a specific but limited pu…

I believe the formalisms can be constructed by something fuzzy. Humans are fuzzy; they create imperefect formalisms that work until they break, and then they're abandoned or adapted. I don't see how LLMs are significantly different. I don't think the formalisms are an "other". I believe they could be tools, both leveraged and maintained by the LLM, in much the same way as most software engineers, when faced with a tr…

I think AI could do this in principle but I haven't seen a convincing demonstration or argument that Transformer based LLMs can do it.

I believe what makes the current Transformer based systems different to humans is that they cannot reliably decide to simulate a deterministic machine while linking the individual steps and the outcomes of that application to the expectations and goals that live in the fuzzy parts of our cognitive system. They cannot think about why the outcome is undesirable and what the smallest possible change would be to make it work.

When we ask them to do things like that, they can do _something_, but it is clearly based on having learned how people talk about it rather than actually applying the formalism themselves. That's why their performance drops off a cliff as soon as the learned patterns get too sparse (I'm sure there's a better term for this that any LLM would be able to tell you :)

Before developing new formalisms you first have to be able to reason properly. Reasoning requires two things. Being able to learn a formalism without examples. And keeping track of the state of a handful of variables while deterministically applying transformation rules.

The fact that the reasoning performance of LLMs drops off a cliff after a number of steps tells me that they are not really reasoning. The 1000th rules based transformation only depending on the previous state of the system should not be more difficult or error prone than the first one, because every step _is_ the first one in a sense. There is no such cliff-edge for humans.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#659

Post by HarHarVeryFunny »

hodgehog11 wrote: post #639
HarHarVeryFunny wrote: post #570

Earlier quoted context omitted.

I don't think he's given up on it. How many decades did it take for neural nets to take off? The reason we're even talking about LeCun today is because he was early in seeing the promise of neural nets and stuck with it through the whole AI winter when most people thought it was a waste of time.

But neural nets were always popular, they just went through phases of hype depending on the capacity of hardware at the time. The only limitation of neural nets at the time was computational power to scale up. AI winters came when other techniques became available that required less compute. Once GPGPU became available, all of that work became immediately viable. No similar limitations exist today for JEPA, to my kno…

Depends on how far back you are going. There was the whole 1969 Minsky Perceptron flap where he said ANNs (i.e Perceptrons) were useless because they can't learn XOR (and no-one at the time knew how to train multi-layer ANNs), which stiffled ANN research and funding for a while. It would then be almost 20 years until the 1986 PDP handbook published LeCun and Hinton's rediscovery of backpropagation as a way to train multi-layer ANNs thereby making them practical.

The JEPA parallel is just that it's not a popular/mainstream approach (at least in terms of well funded research), but may eventually win out over LLMs in the long term. Modern GPUs provide plenty of power for almost any artifical brain type approach, but of course are expensive at scale, so lack of funding can be a barrier in of itself.

Re: Yann LeCun to depart Meta and launch AI startup focused on 'world models'

#660

Post by tedsanders »

qsort wrote: post #64
tedsanders wrote: post #36

Earlier quoted context omitted.

I'm working toward AGI. I hope AGI can be used to automate work and make life easier for people.

>> non-$$ logic [...] aside from misanthropy > I hope AGI can be used to automate work You people need a PR guy, I'm serious. OpenAI is the first company I've ever seen that comes across as actively trying to be misanthropic in its messaging. I'm probably too old-fashioned, but this honestly sounds like Marlboro launching the slogan "lung cancer for the weak of mind".

Tractors automate parts of farmwork. Dishwashers automate parts of dishwashing. Computers automate parts of accounting. People seem to like paying for things that make their lives easier. If people don't like something, they are free to not buy it.
Post reply on HN