261–270 of 680 posts
From the outside, it always looked like they gave LeCun just barely enough compute for small scale experiments. They'd publish a promising new paper, show it works at a small scale, then not use it at all for any of their large AI runs. I would have loved to see a VLM utilizing JEPA for example, but it simply never happened.
Earlier quoted context omitted.
In industry research, someone in a chief position like LeCun should know how to balance long-term research with short-term projects. However, for whatever reason, he consistently shows hostility toward LLMs and engineering projects, even though Llama and PyTorch are two of the most influential projects from Meta AI. His attitude doesn’t really match what is expected from a Chief position at a product company like Fac…
Product companies with deprioritized R&D wings are the first ones to die.
Earlier quoted context omitted.
In industry research, someone in a chief position like LeCun should know how to balance long-term research with short-term projects. However, for whatever reason, he consistently shows hostility toward LLMs and engineering projects, even though Llama and PyTorch are two of the most influential projects from Meta AI. His attitude doesn’t really match what is expected from a Chief position at a product company like Fac…
Yann was never a good fit for Meta.
Best of luck to LeCun. I hope by World Model's he means embodied AI or humanoid robots. We'll have to wait and see.
Earlier quoted context omitted.
Maybe it's time for Bell Labs 2? I guess everyone is racing towards AGI in a few years or whatever so it's kind of impossible to cultivate that environment.
The Bell Labs we look back on was only the result of government intervention in the telecom monopoly. The 1956 consent decree forced Bell to license thousands of its patents, royalty free, to anyone who wanted to use them. Any patent not listed in the consent decree was to be licensed at "reasonable and nondiscriminatory rates." The US government basically forced AT&T to use revenue from its monopoly to do fundamenta…
Earlier quoted context omitted.
LeCun always said that LLMs do not lead to AGI.
Can anyone explain to me the non-$$ logic for one working towards AGI, aside from misanthropy? The only other thing I can imagine is not very charitable: intellectual greed. It can't just be that, can it? I genuinely don't understand. I would love to be educated.
Cure all disease?
Stop aging?
End material scarcity?
It's completely fair to expect that these are all twisted monkey's paw scenarios that turn out dystopian, but being unable to understand any positive motivations for the creation of AGI seems a bit far fetched.
Earlier quoted context omitted.
Maybe it's time for Bell Labs 2? I guess everyone is racing towards AGI in a few years or whatever so it's kind of impossible to cultivate that environment.
[flagged]
"Why Bell Labs Worked" [1]
"The Influence of Bell Labs" [2]
"Bringing back the golden days of Bell Labs" [3]
"Remembering Bell Labs as legendary idea factory prepares to leave N.J. home" [4] or
"Innovation and the Bell Labs Miracle" [5]
interesting too.
[1] https://news.ycombinator.com/item?id=43957010 [2] https://news.ycombinator.com/item?id=42275944 [3] https://news.ycombinator.com/item?id=32352584 [4] https://news.ycombinator.com/item?id=39077867 [5] https://news.ycombinator.com/item?id=3635489
Earlier quoted context omitted.
These are the types that want academic freedom in a cut-throat industry setup and conversely never fit into academia because their profiles and growth ambitions far exceed what an academic research lab can afford (barring some marquee names). It's an unfortunate paradox.
Maybe it's time for Bell Labs 2? I guess everyone is racing towards AGI in a few years or whatever so it's kind of impossible to cultivate that environment.
Earlier quoted context omitted.
Can you explain this “world model” concept to me? How do you actually interface with a model like this?
He is one of these people who think that humans have a direct experience of reality not mediated by as Alan Kay put it three pounds of oatmeal. So he thinks a language model can not be a world model. Despite our own contact with reality being mediated through a myriad of filters and fun house mirror distortions. Our vision transposes left and right and delivers images to our nerves upside down, for gawd’s sake. He im…
your whole reasoning is neither here not there, and attacking a straw man - YLC for sure knows that human experience of reality is heavily modified and distorted
but he also knows, and I'd bet he's very right on this, that we don't "sip reality through a narrow straw of tokens/words", and that we don't learn "just from our/approved written down notes", and only under very specific and expensive circumstances (training runs)
anything closer to more-direct-world-models (as LLMs are ofc at a very indirect level world models) has very high likelihood of yielding lots of benefits
Earlier quoted context omitted.
Can you explain this “world model” concept to me? How do you actually interface with a model like this?
He is one of these people who think that humans have a direct experience of reality not mediated by as Alan Kay put it three pounds of oatmeal. So he thinks a language model can not be a world model. Despite our own contact with reality being mediated through a myriad of filters and fun house mirror distortions. Our vision transposes left and right and delivers images to our nerves upside down, for gawd’s sake. He im…
The mental model this person has of this feed of words is what an LLM at best has (but human model likely much richer since they have a brain, not just a transformer). No real-world experience or grounding, therefore no real-world model. The only model they have is of the world they have experience with - a world of words.