Ask HN: Any insider takes on Yann LeCun's push against current architectures? • Hacker News Board

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#321

Post by CyberDildonics » Sun, Mar 16, 2025, 4:50 PM UTC

YeGoblynQueenne wrote: post #309

throw310822 wrote: post #159

Earlier quoted context omitted.

I don't get it. 1) Yes it's true, learning from text is very hard. But LLMs are multimodal now. 2) That "size of a lion" paper is from 2019, which is a geological era from now. The SOTA was GPT2 which was barely able to spit out coherent text. 3) Have you tried asking a mouse to play chess or reason its way through some physics problem or to write some code? I'm really curious in which benchmark are mice surpassing c…

Oh mice can solve a plethora of physics problems before it's time for breakfast. They have to navigate the, well, physical world, after all. I'm also really curious what benchmarks LLMs have passed that include surviving without being eaten by a cat, or a gull, or an owl, while looking for food to survive and feed one's young in an arbitrary environment chosen from urban, rural, natural etc, at random. What's ChatGPT…

Oh a rock rolling down a hill is, well, navigating the physical world. Is it well, solving physics problem?

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#322

Post by throw310822 » Sun, Mar 16, 2025, 4:57 PM UTC

YeGoblynQueenne wrote: post #317

throw310822 wrote: post #313

Earlier quoted context omitted.

> mice can solve a plethora of physics problems before it's time for breakfast Ah really? Which ones? And nope, physical agility is not "solving a physics problem", otherwise a soccer players and figure skaters would all have PhDs, which doesn't seem to be the case. I mean, an automated system that solves equations to keep balance is not particularly "intelligent". We usually call intelligence the ability to solve ge…

>> Ah really? Which ones? And nope, physical agility is not "solving a physics problem", otherwise a soccer players and figure skaters would all have PhDs, which doesn't seem to be the case. Yes, everything that has to do with navigating physical reality, including, but not restricted to physical agility. Those are physics problems that animals, including humans, know how to solve and, very often, we have no idea how…

> Those are physics problems that animals, including humans, know how to solve

No, those are problems that animals and humans solve, not know how to solve. I'm not the greatest expert of biochemistry that ever lived because of what goes on in my cells.

Now, I understand perfectly well the argument that "even small animals do things that our machines cannot do". That's been indisputably true for a long time. Today, it seems that the be more a matter of embodiment and speed of processing rather than a level of intelligence out of our reach. We already have machines that understand natural language perfectly well and display higher cognitive abilities than any other animal- including abstract reasoning, creating and understanding metaphors, following detailed instructions, writing fiction, etc.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#323

Post by famouswaffles » Sun, Mar 16, 2025, 5:50 PM UTC

chongli wrote: post #320

famouswaffles wrote: post #315

Earlier quoted context omitted.

>The book is not infinite, it's flawed. Oh and the human book is surely infinite and unflawed right ? >we keep bumping into the rough edges of LLMs with their hallucinations and faulty reasoning Both things humans also do in excess The Chinese Room is nonsensical. Can you point to any part of your brain that understands English ? I guess you are a Chinese Room then.

Humans have the ability to admit when they do not know something. We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this. They either have the right answer in the book or they make up nonsense (hallucinate). And they do not even know which one they’re doing!

>Humans have the ability to admit when they do not know something.

No not really. It's not even rare that a human confidently says and believes something and really has no idea what he/she's talking about.

>We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this

Yeah they can. And they can do it much better than chance. They just don't do it as well as humans.

>And they do not even know which one they’re doing!

There's plenty of research that suggests this is the case.

https://news.ycombinator.com/item?id=41418486

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#324

Post by numba888 » Sun, Mar 16, 2025, 6:16 PM UTC

rglover wrote: post #54

Not an ML researcher, but implementing these systems has shown this opinion to be correct. The non-determinism of LLMs is a feature, not a bug that can be fixed. As a result, you'll never be able to get 100% consistent outputs or behavior (like you hypothetically can with a traditional algorithm/business logic). And that has proven out in usage across every model I've worked with. There's also an upper-bound problem…

> where every LLM hits some arbitrary amount of context that causes it to "lose focus" and develop a sort of LLM ADD.

Humans brains have the same problem. As any intelligence probably. Solution for this is structural thinking. One piece at a time, often top-down. Educated humans do it, LLM can be orchestrated to do it too. Effective context window will be limited even though some claim millions of tokens.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#325

Post by numba888 » Sun, Mar 16, 2025, 6:20 PM UTC

choilive wrote: post #112

sockaddr wrote: post #104

Earlier quoted context omitted.

Also, a brain evolved to be a stable compute platform in body that finds itself in many different temperature and energy regimes. And the brain can withstand and recover from some pretty severe damage. So I'd suspect an intelligence that is designed to run in a tighter temp/power envelope with no need for recovery or redundancy could be significantly more efficient than our brain.

Most brain damage would not be considered in the realm of what most people would consider "recoverable".

In some cases it doesn't recover even without physical or chemical damage. Psychiatric clinics are full of this stories.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#326

Post by numba888 » Sun, Mar 16, 2025, 6:23 PM UTC

inimino wrote: post #301

zamubafoo wrote: post #120

Earlier quoted context omitted.

Honest question: Given that the only wide consensus of anything approaching general intelligence are humans and that humans are biological systems that have evolved in physical reality, is there any arguments that better efficiency is even possible without relying on leveraging the nature of reality? For example, analog computers can differentiate near instantly by leveraging the nature of electromagnetism and you ca…

Inference is already wasteful (compared to humans) but training is absurd. There's strong reason to believe we can do better (even prior to having figured out how).

That would mean with current resources AI can get so much more intelligent than humans, right? Aren't you scared?

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#327

Post by [deleted] » Sun, Mar 16, 2025, 7:02 PM UTC

golergka wrote: post #223

ThinkBeat wrote: post #146

Earlier quoted context omitted.

No. > The sun feels hot on your skin. No matter how many times you read that, you cannot understand what the experience is like. > You can read a book about Yoga and read about the Tittibhasana pose But by just reading you will not understand what it feels like. And unless you are in great shape and with greate balance you will fail for a while before you get it right. (which is only human). I have read what shooting…

> No matter how many times you read that, you cannot understand what the experience is like. OK, so you don't have qualia. But if know all the data needed to complete any tasks that can be related to this knowledge, does it matter?

[deleted]

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#328

Post by kerkeslager » Sun, Mar 16, 2025, 7:07 PM UTC

Lerc wrote: post #293

kerkeslager wrote: post #285

Earlier quoted context omitted.

I think it's more fundamental than that. If you start saying "it thinks" in regards to an LLM, you're wrong. LLMs don't think, they pattern match fuzzily. If the training data contained a bunch of answers to questions which were simply "I don't know", you could get an LLM to say "I don't know" but that's still not actually a concept of not knowing. That's just knowing that the answer to your question is "I don't know…

>I think it's more fundamental than that. If you start saying "it thinks" in regards to an LLM, you're wrong. LLMs don't think, they pattern match fuzzily. I'm not sure if this objection is terribly helpful. We use terms like think and want to describe processes that are clearly not involve any form of understanding. Electrons do not have motivations but they 'want' to go to a lower energy level in an atom. You can h…

> We use terms like think and want to describe processes that are clearly not involve any form of understanding.

...and that's why so many people are confused about what's going on with LLMs: sloppy, ambiguous use of language.

> In the Karpathy video I mentioned he talks about how researches found that models did have an internal representation of not knowing, but that the fine tuning was restricting it to providing answers. Giving it fine-tuning examples where it said "I don't know" for information that they knew the model didn't know.

This is why I included the HTTP example: this is simply telling it to parrot the phrase "I don't know"--it doesn't understand that it doesn't know. From the LLM's perpective, it "knows" that the answer is "I don't know". It's returning a 200 OK that says "I don't know" rather than returning a 404.

Do you understand the distinction I'm making here?

> I would agree that models do not have any in-depth understanding of what lack of knowledge actually is. On the other hand I would also think that this also applies to humans, most people are not philosophers.

The average (non-programmer) human, when asked to write a "Hello, world" program, can definitely say they don't know how to program. And unlike the LLM, the human knows that this is different from answering the question. The LLM, in contrast thinks it is answering the question when it says "I don't know"--it thinks "I don't know" is the correct answer.

Put another way, a human can distinguish between responses to these two questions, whereas an LLM can't:

1. What is my grandmother's maiden name?

2. What is the English translation of the Spanish phrase, "No sé."?

In the first question, you don't know the answer unless you are quite creepy; in the second case you do (or can find out easily). But the LLM tuned to answer I don't know thinks it knows the answer in both cases, and thinks the answer is the same.

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#329

Post by chongli » Sun, Mar 16, 2025, 8:14 PM UTC

famouswaffles wrote: post #323

chongli wrote: post #320

Earlier quoted context omitted.

Humans have the ability to admit when they do not know something. We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this. They either have the right answer in the book or they make up nonsense (hallucinate). And they do not even know which one they’re doing!

>Humans have the ability to admit when they do not know something. No not really. It's not even rare that a human confidently says and believes something and really has no idea what he/she's talking about. >We say “sorry, I don’t know, let me get back to you.” LLMs cannot do this Yeah they can. And they can do it much better than chance. They just don't do it as well as humans. >And they do not even know which one th…

No not really. It's not even rare that a human confidently says and believes something and really has no idea what he/she's talking about.

Like you’re doing right now? People say “I don’t know” all the time. Especially children. That people also exaggerate, bluff, and outright lie is not proof that people don’t have this ability.

When people are put in situations where they will be shamed or suffer other social stigmas for admitting ignorance then we can expect them to be less than candid.

As for your links to research showing that LLMs do possess the ability of introspection, I have one question: why have we not seen this in consumer-facing tools? Are the LLMs afraid of social stigma?

Re: Ask HN: Any insider takes on Yann LeCun's push against current architectures?

#330

Post by adamnemecek » Sun, Mar 16, 2025, 10:37 PM UTC

nickpsecurity wrote: post #316

adamnemecek wrote: post #305

Earlier quoted context omitted.

How many of these were actually addressing scaling EBMs though? I'm guessing none.

Including yours. Your landing page has no architecture, model, or performance comparisons. It's non-existent. You need something more tangible for us to believe in. Remember that scientific method requires us to reject everything by default. Only after rigorous review of a working theory or prototype do we treat it as truth. Build what you want us to believe in. Let us see it smoke the competing models of similar siz…

> Remember that scientific method requires us to reject everything by default

You are nowhere near as smart as you think you are. You are a STEMlord who has never produced any new knowledge who just repeats some platitudes. People doing actual research do not talk like this.

You might benefit from watching this video https://x.com/styx_boatman/status/1811820327552315805

Our work is very much work in progress. I mentioned it because we have a very promising path to scaling EBMs and I wanted to have a convo about it.

If you were actually curious and you actually cared about my claims, you would have asked some concrete followup questions. You responded with the dumbest cliches, so I will ignore your comments.