Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun raises $1B to build AI that understands the physical world

wired.com

Post reply on HN
Print view View original

281–290 of 529 posts

Re: Yann LeCun raises $1B to build AI that understands the physical world

#281

Post by scarmig »

nradov wrote: post #237
awakeasleep wrote: post #222

Earlier quoted context omitted.

If I'm understanding you, it seems like you're struck by hindsight bias. No one knew the miasma theory was wrong... it could have been right! Only with hindsight can we say it was wrong. Seems like we're in the same situation with LLMs and AGI.

The miasma theory of disease was "not even wrong" in the sense that it was formulated before we even had the modern scientific method to define the criteria for a theory in the first place. And it was sort of accidentally correct in that some non-infectious diseases are caused by airborne toxins.

Plenty of scientific authorities believed in it through the 19th century, and they didn't blindly believe it: it had good arguments for it, and intelligent people weighed the pros and cons of it and often ended up on the side of miasma over contagionism. William Farr was no idiot, and he had sophisticated statistical arguments for it. And, as evidence that it was a scientific theory, it was abandoned by its proponents once contagionism had more evidence on its side.

It's only with hindsight that we think contagionism is obviously correct.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#282

Post by ttoinou »

lotsofpulp wrote: post #269
ttoinou wrote: post #87

Earlier quoted context omitted.

No he said something like “well yes, only for the parts of profits made in France”

Why would it be any other way?

French people have this pipe dream all others french people to pay 75% of what they produce worldwide to pay for their retreats, hospital, useless schools system and all theirs “comité Théodule”

Re: Yann LeCun raises $1B to build AI that understands the physical world

#284

Post by chriskanan »

I had lunch with Yann last August, about a week after Alex Wang became his "boss." I asked him how he felt about that, and at the time he told me he would give it a month or two and see how it goes, and then figure out if he should stay or find employment elsewhere. I told him he ought to just create his own company if he decides to leave Meta to chase his own dream, rather than work on the dream's of others.

That said, while I 100% agree with him that LLM's won't lead to human-like intelligence (I think AGI is now an overloaded term, but Yann uses it in its original definition), I'm not fully on board with his world model strategy as the path forward.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#285

Post by squeegmeister »

staticman2 wrote: post #97
energy123 wrote: post #38

Earlier quoted context omitted.

I don't understand why online learning is that necessary. If you took Einstein at 40 and surgically removed his hippocampus so he can't learn anything he didn't already know (meaning no online learning), that's still a very useful AGI. A hippocampus is a nice upgrade to that, but not super obviously on the critical path.

> If you took Einstein at 40 and surgically removed his hippocampus so he can't learn anything he didn't already know (meaning no online learning), that's still a very useful AGI. I like how people are accepting this dubious assertion that Einstein would be "useful" if you surgically removed his hippocampus and engaging with this. It also calls this Einstein an AGI rather than a disabled human???

Hypotheticals fear him

Re: Yann LeCun raises $1B to build AI that understands the physical world

#286

Post by sofixa »

re-thc wrote: post #54
Brajeshwar wrote: post #51

Earlier quoted context omitted.

There seem to be other news articles mentioning that they are setting up in Singapore as their base. https://www.straitstimes.com/business/ai-godfather-raises-1-...

> they are setting up in Singapore as their base Europe in general has been tightening up their rules / taxes / laws around startups / companies especially tech and remote. It's been less friendly. these days.

> Europe in general has been tightening up their rules / taxes / laws around startups / companies especially tech and remote.

Like? Care to provide any specific examples? "Europe" is a continent composed of various countries, most of which have been doing a lot to make it easier for startups and companies in general.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#287

Post by sofixa »

crystal_revenge wrote: post #219
gandalfstoe wrote: post #206

Earlier quoted context omitted.

Right, they really need a military industrial complex to be "competitive" :eyeroll. Are you suggesting regressing to the stone age?

Europe doesn't want to be reliant (understandably) on the US military for defense, because if they are, as Trump has demonstrated, they will be pressured to make concessions not in their interests. The need for a military is tightly coupled with the EU's need for energy. You can see this in the immediate impact that the war in Iran has had on Germany's natural gas prices [0]. But already unable to defend itself from…

> But already unable to defend itself from Russia, EU countries

I'm sorry, but this is just crazy talk. Russia cannot enforce its will on Ukraine, one of the poorest and most corrupt countries in Europe, with a (at time of invasion) relatively small and underequipped army. Yes it has grown through conscription, has been equipped by foreign and domestic supplies, has made some brilliant advances in tech and tactics... but when it was attacked, it was weak. And Russia lost its best troops and equipment failing to defat that.

Why would anyone think that the Russia that cannot defeat Ukraine would fare better against Poland? Let alone French warning strike nukes, or French, British, German troops and planes and what not.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#288

Post by HarHarVeryFunny »

sothatsit wrote: post #241
HarHarVeryFunny wrote: post #227

Earlier quoted context omitted.

I think "world models" is the wrong thing to focus on when contrasting the "animal intelligence" approach (which is what LeCun is striving for) with LLMs, especially since "world model" means different things to different people. Some people would call the internal abstractions/representations that an LLM learns during training a "world model" (of sorts). The fundamental problem with today's LLMs that will prevent th…

RL on LLMs has changed things. LLMs are not stuck in continuation predicting territory any more. Models build up this big knowledge base by predicting continuations. But then their RL stage gives rewards for completing problems successfully. This requires learning and generalisation to do well, and indeed RL marked a turning point in LLM performance. A year after RL was made to work, LLMs can now operate in agent har…

RL adds a lot of capability in the areas where it can be applied, but I don't think it really changes the fundamental nature of LLMs - they are still predicting training set continuations, but now trying to predict/select continuations that amount to reasoning steps steering the output in a direction that had been rewarded during training.

At the end of the day it's still copying, not learning.

RL seems to mostly only generalize in-domain. The RL-trained model may be able to generate a working C compiler, but the "logical reasoning" it had baked into it to achieve this still doesn't stop it from telling you to walk to the car wash, leaving your car at home.

There may still be more surprises coming from LLMs - ways to wring more capability out of them, as RL did, without fundamentally changing the approach, but I think we'll eventually need to adopt the animal intelligence approach of predicting the world rather than predicting training samples to achieve human-like, human-level intelligence (AGI).

Re: Yann LeCun raises $1B to build AI that understands the physical world

#289

Post by lxgr »

10xDev wrote: post #52
andy12_ wrote: post #30

Earlier quoted context omitted.

I don't understand this view. How I see it the fundamental bottleneck to AGI is continual learning and backpropagation. Models today are static, and human brains don't learn or adapt themselves with anything close to backpropagation. World models don't solve any of these problems; they are fundamentally the same kind of deep learning architectures we are used to work with. Heck, if you think learning from the world i…

The fact that models aren't continually updating seems more like a feature. I want to know the model is exactly the same as it was the last time I used it. Any new information it needs can be stored in its context window or stored in a file to read the next it needs to access it.

Persistent memory through text in the context window is a hack/workaround.

And generally:

> I want to know the model is exactly the same as it was the last time I used it.

What exactly does that gain you, when the overall behavior is still stochastic?

But still, if it's important to you, you can get the same behavior by taking a model snapshot once we crack continuous learning.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#290

Post by HarHarVeryFunny »

libraryofbabel wrote: post #273
sothatsit wrote: post #241

Earlier quoted context omitted.

RL on LLMs has changed things. LLMs are not stuck in continuation predicting territory any more. Models build up this big knowledge base by predicting continuations. But then their RL stage gives rewards for completing problems successfully. This requires learning and generalisation to do well, and indeed RL marked a turning point in LLM performance. A year after RL was made to work, LLMs can now operate in agent har…

Thanks for saying this. It never ceases to amaze me how many people still talk about LLMs like it’s 2023, completely ignoring the RLVR revolution that gave us models like Opus that can one-shot huge chunks of works-first-time code for novel use cases. Modern LLMs aren’t just trained to guess the next token, they are trained to solve tasks .

Forget 2023 - the advances in coding ability in just last 2-months are amazing. But, they are still not AGI, and it is almost certainly going to take more than just a new training regime such as RL to get there. Demis Hassabis estimates we need another 2-3 "transformer-level" discoveries to get there.
Post reply on HN