Off topic, in case anyone wants to reject cookies, click the underlined "228" in the popup's:
> We, and our 228 partners use cookies
And then you'll see a "reject all" button. Can't make this up.
Yann LeCun raises $1B to build AI that understands the physical world
461–470 of 529 posts
Re: Yann LeCun raises $1B to build AI that understands the physical world
#462> But this is not an applied AI company. There is absolutely no doubt about Yann's impact on AI/ML, but he had access to many more resources in Meta, and we didn't see anything. It could be a management issue, though, and I sincerely wish we will see more competition, but from what I quoted above, it does not seem like it. Understanding world through videos (mentioned in the article), is just what video models have a…
this is absolutely an applied ai company, the only question is whether the applied AI will be subordinated to the research
Re: Yann LeCun raises $1B to build AI that understands the physical world
#463Justifiable. There are a lot more degrees of freedom in world models. LLMs are fundamentally capped because they only learn from static text -- human communications about the world -- rather than from the world itself, which is why they can remix existing ideas but find it all but impossible to produce genuinely novel discoveries or inventions. A well-funded and well-run startup building physical world models (ground…
> LLMs are fundamentally capped because they only learn from static text -- human communications about the world -- rather than from the world itself, which is why they can remix existing ideas but find it all but impossible to produce genuinely novel discoveries or inventions. This seems wrong to me on a few levels. First, there is no way to "experience the world directly," all experience is indirect, and language i…
Re 1: You experience the world in real time (or close enough) via your senses, which combine to form a spatiotemporal sense: A sense of being a bounded entity in space and time. The LLM has none of that. They experience the world via stale old text and text derivatives.
Re 2: There's something tremendous in the fact, staring us right in the face, that LLMs are unable to meaningfully contribute to academic/medical research. I'm not saying that they need to perform on the level of a one-in-a-million Maxwell, DaVinci, or whatever. But as Dwarkesh asked one year ago: "What do you make of the fact that these things have basically the entire corpus of human knowledge memorized and they haven't been able to make a single new connection that has led to a discovery?"
Re 3: Sure, you can hold it by the hand and spoonfeed it. You can also create for it a mirror reality which doesn't exist, which is pure fiction. Given how limited these systems are, I don't suppose it makes much of a difference. There's no way for it to tell. The "human in the loop" is its interaction with the world. And a pale, meager interaction it is.
Re 4: Static, old images/video that they were trained on some months ago. That, too, is no way of interacting with the world.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#464Earlier quoted context omitted.
> LLMs are fundamentally capped because they only learn from static text -- human communications about the world -- rather than from the world itself, which is why they can remix existing ideas but find it all but impossible to produce genuinely novel discoveries or inventions. This seems wrong to me on a few levels. First, there is no way to "experience the world directly," all experience is indirect, and language i…
Re 1: You experience the world in real time (or close enough) via your senses, which combine to form a spatiotemporal sense: A sense of being a bounded entity in space and time. The LLM has none of that. They experience the world via stale old text and text derivatives. Re 2: There's something tremendous in the fact, staring us right in the face, that LLMs are unable to meaningfully contribute to academic/medical res…
> Re 1: You experience the world in real time (or close enough) via your senses, which combine to form a spatiotemporal sense: A sense of being a bounded entity in space and time. The LLM has none of that. They experience the world via stale old text and text derivatives.
It's not clear to me that this is a fundamental limitation. If you provide LLMs with a news feed, it's closer to real-time. You can incrementally get closer than that in very obvious ways.
> Re 2: There's something tremendous in the fact, staring us right in the face, that LLMs are unable to meaningfully contribute to academic/medical research. I'm not saying that they need to perform on the level of a one-in-a-million Maxwell, DaVinci, or whatever. But as Dwarkesh asked one year ago: "What do you make of the fact that these things have basically the entire corpus of human knowledge memorized and they haven't been able to make a single new connection that has led to a discovery?"
LLMs have been around for a very short time. It wouldn't surprise me if researchers have used them to make discoveries. If they haven't, they will soon. Then there's a question about attribution...if you're a researcher and you use an LLM to discover something, do you give it credit? Or is it just a tool? There's a long, long history of researchers being less than honest how they made some discovery.
> Re 3: Sure, you can hold it by the hand and spoonfeed it. You can also create for it a mirror reality which doesn't exist, which is pure fiction. Given how limited these systems are, I don't suppose it makes much of a difference. There's no way for it to tell. The "human in the loop" is its interaction with the world. And a pale, meager interaction it is.
Our perception of reality is meager too. You can easily imagine how an LLM could be "plugged in" to reality. Again nothing fundamental here.
> Re 4: Static, old images/video that they were trained on some months ago. That, too, is no way of interacting with the world.
No, you can send an LLM a video/image and it can "understand it". It's not perfect but, like I said, the technology is already here to project video data into something the LLMs can interact with.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#465[dead]
Re: Yann LeCun raises $1B to build AI that understands the physical world
#466Earlier quoted context omitted.
Sure, Claude and other SOTA LLMs do generate about 90% of my code but I feel like we are not closer to solving the last 10% than we were a year ago in the days of Claude 3.7. It can pretty reliably get 90% there and then I can either keep prompting it to get the rest done or just do it manually which is quite often faster.
It's interesting that people don't seem to think the likely outcome might be... capital and labour. Not capital alone. You see this in construction - the capital is used for certain things and is operated by labour.
We're certainly in the "capital/robot + labor" phase of AI at the moment, which Dario Amodei is referring to as the "centaur" (half horse, half human) phase, and expects to be very short lived.
Eventually (maybe taking a lot longer than a lot of people expect and/or are hoping for) we'll achieve full human-equivalent AI, at which point you won't NEED a centaur approach - the mechanical horse will be capable of doing ALL non-physical work by itself, but that doesn't mean this is how this will actually play out. If we do end up heading for some dystopian "Soylent Green" type future where most humans are unemployed, surviving poorly on government handouts, then I expect there would eventually be riots and uprising that would push back against it. It also just doesn't work - you can't create profits without customers, and customers need money to buy what you're selling.
Part of why we may (and hopefully will) continue to see humans, from CEO on down, still working when they could be replaced with AI, is that even "AGI", which we've yet to achieve, doesn't mean human-like - it's really just focusing on intelligence. Creating an actual remote-worker replacement requires more than just automating the intelligent decision-making part of a human (the "AGI" part) - it also requires the human/social/emotional part, which will take longer, and there may not even be any desire to push for that. I think people maybe discount how much of being a successful member of a team is based around human soft skills, our ability to understand and interact with each other, not just raw intellectual capacity, and certainly at this point in time corporate success is still very much "who you know, not what you know".
Re: Yann LeCun raises $1B to build AI that understands the physical world
#467Earlier quoted context omitted.
Agents have the ability of continual learning.
Putting stuff you have learned into a markdown file is a very "shallow" version of continual learning. It can remember facts, yes, but I doubt a model can master new out-of-distribution tasks this way. If anything, I think that Google's Titans[1] and Hope[2] architectures are more aligned with true continual learning (without being actual continual learning still, which is why they call it "test-time memorization").…
It really reminds me of the movie Memento - it has to constantly put notes down to remember who it is and what it should do after waking up without memory every n minutes.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#468Earlier quoted context omitted.
> The story that humans have access to some pure deductive engine and LLMs are just faking it with statistics might be flattering to humans more than it’s accurate. Your point rings true with most human reasoning most of the time. Still, at least some humans do have the capability to run that deductive engine, and it seems to be a key part (though not the only part) of scientific and mathematical reasoning. Even info…
The fact that humans can learn to do X, sometimes well, often badly, and while many don’t, strongly supports the conjecture that X is not how they naturally do things. I can perform symbolic calculations too. But most people have limited versions of this skill, and many people who don’t learn to think symbolically have full lives. I think it is fair to say humans don’t naturally think in formal or symbolic reasoning…
I don't disagree with any of that. My comment was only in relation to the question of human-specific capability that current LLMs may not be able to duplicate. I was not making the value judgments you seem to have read.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#469This is so stupid. AI already understands the physical world. What it can't do is interact with it. There's a hardware bottleneck. It simply isn't responsive.
Re: Yann LeCun raises $1B to build AI that understands the physical world
#470Earlier quoted context omitted.
> LLMs are fundamentally capped because they only learn from static text -- human communications about the world -- rather than from the world itself, which is why they can remix existing ideas but find it all but impossible to produce genuinely novel discoveries or inventions. This seems wrong to me on a few levels. First, there is no way to "experience the world directly," all experience is indirect, and language i…
Re 1: You experience the world in real time (or close enough) via your senses, which combine to form a spatiotemporal sense: A sense of being a bounded entity in space and time. The LLM has none of that. They experience the world via stale old text and text derivatives. Re 2: There's something tremendous in the fact, staring us right in the face, that LLMs are unable to meaningfully contribute to academic/medical res…
> What do you make of the fact that these things have basically the entire corpus of human knowledge memorized and they haven't been able to make a single new connection that has led to a discovery?
If that's what you're experiencing, then you're not asking them the right questions.
If you're at the edge of your field so you're able to judge whether something is novel or not, and you have a direction you'd like the LLM to explore, just ask it. Prompt it to come up with some ideas of how to solve X, or categorize Y, or analyze Z. Encourage it to take ideas from, or find parallels in, closely related or distantly related fields.
You will probably quickly find yourself with a ton of new ideas, of varying quality, in the same way as if you were brainstorming with a colleague.
But they don't work "solo". They need to you guide the conversation. But when you do, they're chock-full of new ideas and connections and discoveries. But again -- just like with people, the quality varies. If you're looking for a good startup idea, you need to sift through hundreds. Similarly if you're looking for an idea of a paper you could publish, there are a lot of hypotheses to sift through. And you're supplying your own expert "good taste" to try to determine what's worth pursuing and developing further, etc.
LLMs don't just magically come up with new proven discoveries unprompted. But they turn out to be fantastic research and idea-generation partners. They excel at combining existing related-but-distant facts and models and interpretations in novel ways.