Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Any insider takes on Yann LeCun's push against current architectures?

news.ycombinator.com

Post reply on HN
Print view View original

211–220 of 343 posts

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#211

Post by kadushka »

iainctduncan wrote: post #208
bravura wrote: post #132

Okay I think I qualify. I'll bite. LeCun's argument is this: 1) You can't learn an accurate world model just from text. 2) Multimodal learning (vision, language, etc) and interaction with the environment is crucial for true learning. He and people like Hinton and Bengio have been saying for a while that there are tasks that mice can understand that an AI can't. And that even have mouse-level intelligence will be a br…

Thanks for articulating this so well. I'm a musician and music/CS phd student, and as a jazz improvisor of advanced skill (30+ years), I'm accutely aware that there are significant areas of intelligence for which linguistic thinking is not only not good enough, but something to be avoided as much as one can (which is bloody hard sometimes). I have found it so frustrating, but hard to figure out how to counter, that t…

Most modern LLMs are multimodal.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#212

Post by trod1234 »

Not an ML researcher, but neither of those ideas are going to work.

The token approach is inherently flawed because the tokens pre-suppose unique meaning when in fact they may not be unique.

Said another way, it lacks properties that would be able to differentiate true from false because the differentiating input isn't included and cannot be derived from the inputs given. This goes to decidability.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#213

Post by csomar »

ThinkBeat wrote: post #146
codenlearn wrote: post #138

Earlier quoted context omitted.

Doesn't Language itself encode multimodal experiences? Let's take this case write when we write text, we have the skill and opportunity to encode the visual, tactile, and other sensory experiences into words. and the fact is llm's trained on massive text corpora are indirectly learning from human multimodal experiences translated into language. This might be less direct than firsthand sensory experience, but potentia…

No. > The sun feels hot on your skin. No matter how many times you read that, you cannot understand what the experience is like. > You can read a book about Yoga and read about the Tittibhasana pose But by just reading you will not understand what it feels like. And unless you are in great shape and with greate balance you will fail for a while before you get it right. (which is only human). I have read what shooting…

All of these "experiences" are encoded in your brain as electricity. So "text" can encode them, though English words might not be the proper way to do it.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#214

Post by ninetyninenine »

bravura wrote: post #132

Okay I think I qualify. I'll bite. LeCun's argument is this: 1) You can't learn an accurate world model just from text. 2) Multimodal learning (vision, language, etc) and interaction with the environment is crucial for true learning. He and people like Hinton and Bengio have been saying for a while that there are tasks that mice can understand that an AI can't. And that even have mouse-level intelligence will be a br…

>1) You can't learn an accurate world model just from text. >2) Multimodal learning (vision, language, etc) and interaction with the environment is crucial for true learning.

LLMs can be trained with multimodal data. Language is only tokens and pixel and sound data can be encoded into tokens. All data can be serialized. You can train this thing on data we can't even comprehend.

Here's the big question. It's clear we need less data then an LLM. But I think it's because evolution has pretrained our brains for this so we have brains geared towards specific things. Like we are geared towards walking, talking, reading, in the same way a cheetah is geared towards ground speed more then it is at flight.

If we placed a human and an LLM in completely unfamiliar spaces and tried to train both with data. Which will perform better?

And I mean completely non familiar spaces. Like let's make it non Euclidean space and only using sonar for visualization. Something totally foreign to reality as humans know it.

I honestly think the LLM will beat us in this environment. We might've succeeded already in creating AGI it's just the G is too much. It's too general so it's learning everything from scratch and it can't catch up to us.

Maybe what we need is to figure out how to bias the AI to think and be biased in the way humans are biased.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#215

Post by tyronehed »

killthebuddha wrote: post #10

I've always felt like the argument is super flimsy because "of course we can _in theory_ do error correction". I've never seen even a semi-rigorous argument that error correction is _theoretically_ impossible. Do you have a link to somewhere where such an argument is made?

As soon as you need to start leaning heavily on error correction, that is an indication that your architecture and solution is not correct. The final solution will need to be elegant and very close to a perfect solution immediately.

You must always keep close to the only known example we have of an intelligence which is the human brain. As soon as you start to wander away from the way the human brain does it, you are on your own and you are not relying on known examples of intelligence. Certainly that might be possible, but since there's only one known example in this universe of intelligence, it seems ridiculous to do anything but stick close to that example, which is the human brain.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#216

Post by QuesnayJr »

wavemode wrote: post #202
throw310822 wrote: post #45

Earlier quoted context omitted.

> there isn't a "I don't have enough information" option. This is due to the fact that LLMs are basically just giant look up maps with interpolation. Have you ever tried telling ChatGPT that you're "in the city centre" and asking it if you need to turn left or right to reach some landmark? It will not answer with the average of the directions given to everybody who asked the question before, it will answer asking you…

That's because, based on the training data, the most likely response to asking for directions is to clarify exactly where you are and what you see. But if you ask it in terms of a knowledge test ("I'm at the corner of 1st and 2nd, what public park am I standing next to?") a model lacking web search capabilities will confidently hallucinate (unless it's a well-known park). In fact, my person opinion is that, therein l…

I tried this just now on Chatbot Arena, and both chatbots asked for more information.

One was GPT 4.5 preview, and one was cohort-chowder (which is someone's idea of a cute code name, I assume).

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#217

Post by Tepix »

pessimizer wrote: post #207
ninetyninenine wrote: post #204

Earlier quoted context omitted.

LLM is just the name. You can encode anything into the "language" including pictures video and sound.

I've always been wondering if anyone is working on using nerve impulses. My first thought when transformers came around was if they could be used for prosthetics, but I've been too lazy to do the research to find anybody working on anything like that, or to experiment myself with it.

When you train a neural net for Donkeycar with camera images plus the joystick commands of the driver, isn't that close to nerve impulses already?

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#218

Post by CamperBob2 »

mystified5016 wrote: post #154
codenlearn wrote: post #138

Earlier quoted context omitted.

Doesn't Language itself encode multimodal experiences? Let's take this case write when we write text, we have the skill and opportunity to encode the visual, tactile, and other sensory experiences into words. and the fact is llm's trained on massive text corpora are indirectly learning from human multimodal experiences translated into language. This might be less direct than firsthand sensory experience, but potentia…

Imagine I give you a text of any arbitrary length in an unknown language with no images. With no context other than the text, what could you learn? If I told you the text contained a detailed theory of FTL travel, could you ever construct the engine? Could you even prove it contained what I told you? Can you imagine that given enough time, you'd recognize patterns in the text? Some sequences of glyphs usually follow…

A few GB worth of photographs of hieroglyphs? OK, you're going to need a Rosetta Stone.

A few PB worth? Relax, HAL's got this. When it comes to information, it turns out that quantity has a quality all its own.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#219

Post by wavemode »

QuesnayJr wrote: post #216
wavemode wrote: post #202

Earlier quoted context omitted.

That's because, based on the training data, the most likely response to asking for directions is to clarify exactly where you are and what you see. But if you ask it in terms of a knowledge test ("I'm at the corner of 1st and 2nd, what public park am I standing next to?") a model lacking web search capabilities will confidently hallucinate (unless it's a well-known park). In fact, my person opinion is that, therein l…

I tried this just now on Chatbot Arena, and both chatbots asked for more information. One was GPT 4.5 preview, and one was cohort-chowder (which is someone's idea of a cute code name, I assume).

I tried this just now on Chatbot Arena, and both chatbots very confidently got the name of the park wrong.

Perhaps you thought I meant "1st and 2nd" literally? I was just using those as an example so I don't reveal where I live. You should use actual street names that are near a public park, and you can feel free to specify the city and state.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#220

Post by petesergeant »

bravura wrote: post #132

Okay I think I qualify. I'll bite. LeCun's argument is this: 1) You can't learn an accurate world model just from text. 2) Multimodal learning (vision, language, etc) and interaction with the environment is crucial for true learning. He and people like Hinton and Bengio have been saying for a while that there are tasks that mice can understand that an AI can't. And that even have mouse-level intelligence will be a br…

> LeCun isn't saying that LLMs aren't useful. He's just concerned with bigger problems, like AGI, which he believes cannot be solved purely through linguistic analysis.

It feels like special pleading: surely _this_ will be the problem class that doesn’t fall to “the bitter lesson”.

My intuition is that the main problem with the current architecture is that mapping into tokens causes quantization that a real brain doesn’t have, and lack of plasticity.

I don’t build models, I spend 100% of my time reading and adjusting model outputs though.

Post reply on HN