[flagged]
Yann LeCun raises $1B to build AI that understands the physical world
491–500 of 529 posts
Re: Yann LeCun raises $1B to build AI that understands the physical world
#492It's curious to me why we have no theory of intelligence. By which I mean an actual hard and verified theory, as in physics for gravity, electromagnetism, quantum mechanics. Intelligence is simply not well-understood at a mathematical level. Like medieval engineers, we rely so heavily on experimentation in AI. We have no idea how far away from the human level we actually are. Or how far above the human level we can g…
Re: Yann LeCun raises $1B to build AI that understands the physical world
#493I feel HN comments have been getting hijacked for a long time now by LLM agents. Always so early, very positive, and hard to spot. Some replaced em-dash with --, some replace them with a single dash, some remove them all together. I wonder how much time it is taking from @dang and other moderators helping to maintain this community.
Can you mention some specific examples? If you don't want to post them here, emailing hn@ycombinator.com would be good. We recently promoted the no-generated-comments rule from case law [1] to the site guidelines [2], and we're banning accounts that break it. [1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... [2] https://news.ycombinator.com/newsguidelines.html#generated
I will start sharing them moving forward. I've had several parent threads deleted in the past(rightfully so) because it turned out I was responding to AI. Thank you for staying on top of this.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#494LeCun has been arguing for years that predicting text isn’t enough for real intelligence. This startup is basically his attempt to prove that world models — not bigger LLMs — are the path to AI that can reason about the physical world.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#495Earlier quoted context omitted.
> LLMs are fundamentally capped because they only learn from static text -- human communications about the world -- rather than from the world itself, which is why they can remix existing ideas but find it all but impossible to produce genuinely novel discoveries or inventions. This seems wrong to me on a few levels. First, there is no way to "experience the world directly," all experience is indirect, and language i…
Re 1: You experience the world in real time (or close enough) via your senses, which combine to form a spatiotemporal sense: A sense of being a bounded entity in space and time. The LLM has none of that. They experience the world via stale old text and text derivatives. Re 2: There's something tremendous in the fact, staring us right in the face, that LLMs are unable to meaningfully contribute to academic/medical res…
>Re 2: There's something tremendous in the fact, staring us right in the face, that LLMs are unable to meaningfully contribute to academic/medical research. I'm not saying that they need to perform on the level of a one-in-a-million Maxwell, DaVinci, or whatever. But as Dwarkesh asked one year ago: "What do you make of the fact that these things have basically the entire corpus of human knowledge memorized and they haven't been able to make a single new connection that has led to a discovery?"
This isn't really true so what ? If you really cared and were actually paying attention, you'd see that frontier LLMs have begun to contribute to academic research. There are other impressive results for math as well.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#496Earlier quoted context omitted.
> Whenever I see claims about AGI being reachable through large language models, it reminds me of the miasma theory of disease. Whenever I see people think the model architecture matters much, I think they have a magical view of AI. Progress comes from high quality data, the models are good as they are now. Of course you can still improve the models, but you get much more upside from data, or even better - from inter…
> Of course you can still improve the models, but you get much more upside from data, or even better - from interactive environments. I'm on the contrary believe that the hunt for better data is an attempt to climb the local hill and be stuck there without reaching the global maximum. Interactive environments are good, they can help, but it is just one of possible ways to learn about causality. Is it the best way? I…
>With only instructional materials (a 500-page reference grammar, a dictionary, and ≈400 extra parallel sentences) all provided in context, Gemini 1.5 Pro and Gemini 1.5 Flash are capable of learning to translate from English to Kalamang— a Papuan language with fewer than 200 speakers and therefore almost no online presence—with quality similar to a person who learned from the same materials
Re: Yann LeCun raises $1B to build AI that understands the physical world
#497Earlier quoted context omitted.
We're certainly in the "capital/robot + labor" phase of AI at the moment, which Dario Amodei is referring to as the "centaur" (half horse, half human) phase, and expects to be very short lived. Eventually (maybe taking a lot longer than a lot of people expect and/or are hoping for) we'll achieve full human-equivalent AI, at which point you won't NEED a centaur approach - the mechanical horse will be capable of doing…
you know Amodei is a salesman, right
Yes, very much so! I don't think that was always the case, but something changed about a year ago.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#498Earlier quoted context omitted.
>The fundamental problem with today's LLMs that will prevent them from achieving human level intelligence, and creativity, is that they are trained to predict training set continuations, which creates two very major limitations: I am of the opinion that imagination and creativity comes from emotion, hence a machine that cannot "feel" will never be truly intelligent. One can go ahead and ask, but you are just a lump o…
> I am of the opinion that imagination and creativity comes from emotion How do you see emotion as being necessary for creativity? It sure seems that things like surprise (prediction failure) driven "curiosity" and exploration (I can't predict what will happen if I do X, so let me try) are behind creativity, pushing the boundaries of knowledge and discovering something new. Perhaps you mean artistic creativity rather…
> How do you see emotion as being necessary for creativity?
As its reward. Why should anyone be creative if it does not "feel" good?
>releasing chemicals like adrenaline, dopamine..
That does not explain how it results in a certain kind of "feeling"..
>FWIW I think consciousness is also very mechanical
I think I didn't really do an OK job of why I think otherwise in my comment above...
Re: Yann LeCun raises $1B to build AI that understands the physical world
#499Earlier quoted context omitted.
You can’t really say it is just predicting continuations when it is learning to write proofs for Erdos problems, formalise significant math results, or perform automated AI research. Those are far beyond what you get by just being a copying and re-forming machine, a lot of these problems require sophisticated application of logic. I don’t know if this can reach AGI, or if that term makes any sense to begin with. But…
I think you're conflating mechanism with function/capability. I'm not sure what I wrote that made you conclude that I thought these models are not learning anything from their RL training?! Let me say it again: they are learning to steer towards reasoning steps that during training led to rewards. The capabilities of LLMs, both with and without RL, are a bit counter-intuitive, and I think that, at least in part, come…
This is what you said:
> they are still predicting training set continuations
But this is underselling what they do. Probably a large part of what they predict is learnt from their training set, but RL has added a layer on top that does not just come from just mimicry.
Again, I doubt this is enough for “AGI” but I think that term is not very well-defined to begin with. These models have now shown they are capable of novel reasoning, they just have to be prodded in the right way.
It’s not clear to me that there isn’t scaffolding that can use LLMs to search for novel improvements, like Katpathy’s recent autoresearch. The models, with the help of RL, seem to be getting to the point where this actually works to some extent, and I would expect this to happen in other fields in the next few years as well.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#500Earlier quoted context omitted.
> I am of the opinion that imagination and creativity comes from emotion How do you see emotion as being necessary for creativity? It sure seems that things like surprise (prediction failure) driven "curiosity" and exploration (I can't predict what will happen if I do X, so let me try) are behind creativity, pushing the boundaries of knowledge and discovering something new. Perhaps you mean artistic creativity rather…
> How do you see emotion as being necessary for creativity? As its reward. Why should anyone be creative if it does not "feel" good? >releasing chemicals like adrenaline, dopamine.. That does not explain how it results in a certain kind of "feeling".. >FWIW I think consciousness is also very mechanical I think I didn't really do an OK job of why I think otherwise in my comment above...
> As its reward. Why should anyone be creative if it does not "feel" good?
We've evolved to do things that are helpful for survival.
Things like curiosity and exploration don't need to feel good - it's just our nature. They only need to attract and keep our attention.