Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Yann LeCun raises $1B to build AI that understands the physical world

wired.com

Post reply on HN
Print view View original

481–490 of 529 posts

Re: Yann LeCun raises $1B to build AI that understands the physical world

#481

Post by perfmode »

lich_king wrote: post #243
perfmode wrote: post #232

Earlier quoted context omitted.

Humans are notoriously bad at formal logic. The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. That looks a lot more like pattern matching than rule application. Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. T…

> The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. I've never heard about the Wason selection task, looked it up, and could tell the right answer right away. But I can also tell you why: because I have some familiarity with formal logic and can, in your words, pattern-match the gotcha tha…

I think we're actually closer to agreement than it might seem.

You're right that the Wason task is partly about a mismatch between how "if" works in formal logic and how it works in everyday language. That's a fair point. But I think it actually supports what I'm saying rather than undermining it. If people default to interpreting "if x then y" as "if and only if" based on how language normally works in conversation, that is pattern-matching from familiar context. It's a totally understandable thing to do, and I'm not calling it a cognitive defect. I'm saying it's evidence that our default mode is contextual pattern-matching, not rule application. We agree on the mechanism, we're just drawing different conclusions from it.

Your own experience is interesting too. You got the right answer because you have some background in formal logic. That's exactly what I'd expect. Someone who's practiced in a domain recognizes the pattern quickly. But that's the claim: most reasoning happens within well-practiced domains. Your success on the task doesn't counter the pattern-matching thesis, it's a clean example of it working well.

On the broader point about LLMs having a "tenuous grasp on reality," I hear that, and I don't want to flatten the differences. There probably is something meaningfully different going on with how humans stay grounded. I just think the "humans reason, LLMs pattern-match" framing undersells how much human cognition is also pattern-matching, and that being honest about that is more productive than treating it as a reductionist insult.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#482

Post by bwfan123 »

perfmode wrote: post #476
bwfan123 wrote: post #473

Earlier quoted context omitted.

> But then again, how often do humans actually reason outside their own “training distribution”? Most human insight happens within well-practiced domains. Humans can produce new concepts and then symbolize them for communication purposes. The meaning of concepts is grounded in operational definitions - in a manner that anyone can understand because they are operational, and can be reproduced in theory by anyone. For…

> Humans can produce new concepts and then symbolize them for communication purposes. Sure, but the question is how often this actually happens versus how often people are doing something closer to recombination and pattern-matching within familiar territory. The point was about the base rate of genuine novel reasoning in everyday human cognition, and I don't think this addresses that. > Euclid invented the concepts…

> Formalizing those intuitions is real work, but it's not the same as generating something with no prior basis.

> The fact that Euclid existed doesn't tell us much about what the other billions of humans are doing cognitively on a Tuesday afternoon.

Birds can fly - so, there is some flying intelligence built into their dna. But, are they aware of their skill to be able to create a theory of flight, and then use that to build a plane ? I am just pointing out that intuitions are not enough - the awareness of the intuitions in a manner that can symbolize and operationalize it is important.

> The whole point, drawing on Kahneman, is that most of what we call reasoning is fast associative pattern-matching, and that the slow deliberate stuff is rarer and more error-prone than people assume

David Bessis, in his wonderful book [1] argues that the cognitive actions done by you and I on a tuesday afternoon is the same that mathematicians do - just that we are unaware of it. Also, since you brought up Kahneman, Bessis proposes a System 3 wherein inaccurate intuitions is corrected by precise communication.

[1] Mathematica: A Secret World of Intuition and Curiosity

Re: Yann LeCun raises $1B to build AI that understands the physical world

#483

Post by perfmode »

bwfan123 wrote: post #482
perfmode wrote: post #476

Earlier quoted context omitted.

> Humans can produce new concepts and then symbolize them for communication purposes. Sure, but the question is how often this actually happens versus how often people are doing something closer to recombination and pattern-matching within familiar territory. The point was about the base rate of genuine novel reasoning in everyday human cognition, and I don't think this addresses that. > Euclid invented the concepts…

> Formalizing those intuitions is real work, but it's not the same as generating something with no prior basis. > The fact that Euclid existed doesn't tell us much about what the other billions of humans are doing cognitively on a Tuesday afternoon. Birds can fly - so, there is some flying intelligence built into their dna. But, are they aware of their skill to be able to create a theory of flight, and then use that…

The bird analogy is actually a really good one, but I think it supports a narrower claim than you're making. You're right that the capacity to symbolize and formalize intuitions is a distinct and important thing, separate from just having the intuitions. No argument there. But my point wasn't that symbolization doesn't matter. It was about how often humans actually exercise that capacity in a strong sense versus doing something more like recombination within familiar frameworks. The bird can't theorize flight, agreed. But most humans who can in principle theorize about their intuitions also don't, most of the time. The capacity exists. The base rate of its deployment is the question.

On Bessis, I actually think his argument is more compatible with what I was saying than it might seem. If the cognitive process underlying mathematical reasoning is the same one operating on a Tuesday afternoon, that's an argument against treating Euclid-level formalization as categorically different from everyday cognition. It suggests a continuum rather than a bright line between "pattern matching" and "genuine reasoning." Which is interesting and probably right. But it also means you can't point to Euclid as evidence that humans routinely do something qualitatively beyond what LLMs do. If Bessis is right, then the extraordinary cases and the mundane cases share the same underlying machinery, and the question becomes quantitative (how far along the continuum, how often, under what conditions) rather than categorical.

I'll check out the book though, it sounds like it's making a more careful version of the point than usually gets made in these threads.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#485

Post by HarHarVeryFunny »

qsera wrote: post #475
HarHarVeryFunny wrote: post #227

Earlier quoted context omitted.

I think "world models" is the wrong thing to focus on when contrasting the "animal intelligence" approach (which is what LeCun is striving for) with LLMs, especially since "world model" means different things to different people. Some people would call the internal abstractions/representations that an LLM learns during training a "world model" (of sorts). The fundamental problem with today's LLMs that will prevent th…

>The fundamental problem with today's LLMs that will prevent them from achieving human level intelligence, and creativity, is that they are trained to predict training set continuations, which creates two very major limitations: I am of the opinion that imagination and creativity comes from emotion, hence a machine that cannot "feel" will never be truly intelligent. One can go ahead and ask, but you are just a lump o…

> I am of the opinion that imagination and creativity comes from emotion

How do you see emotion as being necessary for creativity?

It sure seems that things like surprise (prediction failure) driven "curiosity" and exploration (I can't predict what will happen if I do X, so let me try) are behind creativity, pushing the boundaries of knowledge and discovering something new.

Perhaps you mean artistic creativity rather than scientific, in which case we're talking about different things, but I'd agree with you since the goal of much art is to elicit an emotional response in those engaging with it.

I don't think there is anything stopping us from implementing emotions, every bit as real as our own, in some form of artificial life if we want to though. At the end of the day emotion comes down to our primitive brain releasing chemicals like adrenaline, dopamine, etc as a result of certain stimuli, the functioning of our brain/body being affected by those chemicals, and the feedback loop of us then recognizing how our brain/body is operating differently ("I feel sad/exited/afraid" etc). It's all very mechanical.

FWIW I think consciousness is also very mechanical, but it seems somewhat irrelevant to the discussion of intelligence/AGI.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#486

Post by JimSanchez »

Interesting perspective from LeCun. The debate between scaling LLMs versus building systems that understand the physical world seems like one of the big open questions in AI right now. It will be fascinating to see whether “world models” end up complementing LLMs or eventually replacing parts of them.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#487

Post by zahlman »

thih9 wrote: post #461

Off topic, in case anyone wants to reject cookies, click the underlined "228" in the popup's: > We, and our 228 partners use cookies And then you'll see a "reject all" button. Can't make this up.

I just block "all cross-site cookies" in Firefox settings. This "may cause websites to break" but it hasn't affected anything I care about in years.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#488

Post by drdaeman »

nextaccountic wrote: post #359
perfmode wrote: post #232

Earlier quoted context omitted.

Humans are notoriously bad at formal logic. The Wason selection task is the classic example: most people fail a simple conditional reasoning problem unless it’s dressed up in familiar social context, like catching cheaters. That looks a lot more like pattern matching than rule application. Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. T…

> Kahneman’s whole framework points the same direction. Most of what people call “reasoning” is fast, associative, pattern-based. The slow, deliberate, step-by-step stuff is effortful and error-prone, and people avoid it when they can. And even when they do engage it, they’re often confabulating a logical-sounding justification for a conclusion they already reached by other means. Some references on that https://en.w…

In my naive understanding, neither requires any will or consciousness.

S1 is “bare” language production, picking words or concepts to say or think by a fancy pattern prediction. There’s no reasoning at this level, just blabbering. However, language by itself weeds out too obvious nonsense purely statistically (some concepts are rarely in the same room), but we may call that “mindlessly” - that’s why even early LLMs produced semi-meaningful texts.

S2 is a set of patterns inside the language (“logic”), that biases S1 to produce reasoning-like phrases. Doesn’t require any consciousness or will, just concepts pushing S1 towards a special structure, simply backing one keeps them “in mind” and throws in the mix.

I suspect S2 has a spectrum of rigorousness, because one can just throw in some rules (like “if X then Y, not Y therefore not X”) or may do fancier stuff (imposing a larger structure to it all, like formulating and testing a null hypothesis). Either way it all falls down onto S1 for a ultimate decision-making, a sense of what sounds right (allowing us our favorite logical flaws), thus the fancier the rules (patterns of “thought”) the more likely reasoning will be sounder.

S2 doesn’t just rely but is a part of S1-as-language, though, because it’s a phenomena born out (and inside) the language.

Whether it’s willfully “consciously” engaged or if it works just because S1 predicts logical thinking concept as appropriate for certain lines of thinking and starts to involve probably doesn’t even matter - it mainly depends on whatever definition of “will” we would like to pick (there are many).

LLMs and humans can hypothetically do both just fine, but when it comes to checking, humans currently excel because (I suspect) they have a “wider” language in S1, that doesn’t only include word-concepts but also sensory concepts (like visuospatial thinking). Thus, as I get it, the world models idea.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#489

Post by esafak »

nprateem wrote: post #426
voxleone wrote: post #422

I rank with those who think human-like intelligence will require embeddings grounded in multiple physical sensory domains (vision, touch, audio, chemical sensing, etc.) fused into a shared world representation. That seems much closer to how biological intelligence works than text-only models. But if this path succeeds and produces systems with something like genuine understanding or sentience, there’s a deeper questi…

Lol. A lump of metal can't be sentient.

I think the more likely retort will be that we can't be smart, by the AI's standard.

Re: Yann LeCun raises $1B to build AI that understands the physical world

#490

Post by nsjdjdekkddk »

HarHarVeryFunny wrote: post #466
j4k0o wrote: post #459

Earlier quoted context omitted.

It's interesting that people don't seem to think the likely outcome might be... capital and labour. Not capital alone. You see this in construction - the capital is used for certain things and is operated by labour.

We're certainly in the "capital/robot + labor" phase of AI at the moment, which Dario Amodei is referring to as the "centaur" (half horse, half human) phase, and expects to be very short lived. Eventually (maybe taking a lot longer than a lot of people expect and/or are hoping for) we'll achieve full human-equivalent AI, at which point you won't NEED a centaur approach - the mechanical horse will be capable of doing…

you know Amodei is a salesman, right
Post reply on HN