Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs don't do formal reasoning

garymarcus.substack.com

Post reply on HN
Print view View original

121–128 of 128 posts

Re: LLMs don't do formal reasoning

#121

Post by talldayo »

hackinthebochs wrote: post #91
AlienRobot wrote: post #61

Earlier quoted context omitted.

What is moving data round? Isn't everything in a computer moving data around? Do you mean backpropagation or somtehing more specific?

Computation generally is partly moving data around, yes. What transformers do is learn how to move data around in a context-relevant manner. This greatly increases the expressivity of the kinds of computations they can perform over traditional deep nets. https://lilianweng.github.io/posts/2018-06-24-attention/ https://transformer-circuits.pub/2022/in-context-learning-an... https://transformer-circuits.pub/2021/framew…

> What transformers do is learn how to move data around in a context-relevant manner.

This is a misrepresentation of how transformers behave and I think you should double-check the definition before dunking on other people's works.

Re: LLMs don't do formal reasoning

#122

Post by hackinthebochs »

talldayo wrote: post #121
hackinthebochs wrote: post #91

Earlier quoted context omitted.

Computation generally is partly moving data around, yes. What transformers do is learn how to move data around in a context-relevant manner. This greatly increases the expressivity of the kinds of computations they can perform over traditional deep nets. https://lilianweng.github.io/posts/2018-06-24-attention/ https://transformer-circuits.pub/2022/in-context-learning-an... https://transformer-circuits.pub/2021/framew…

> What transformers do is learn how to move data around in a context-relevant manner. This is a misrepresentation of how transformers behave and I think you should double-check the definition before dunking on other people's works.

It's not a misinterpretation. What attention does is discover association matrices which bind locations in the context window, and these associations are context sensitive. But binding locations through an association matrix is an implementation of the concept of routing, which is just moving data.

Also, the link I gave regarding induction heads is explicitly moving data in the context window forward.

Re: LLMs don't do formal reasoning

#123

Post by talldayo »

hackinthebochs wrote: post #122
talldayo wrote: post #121

Earlier quoted context omitted.

> What transformers do is learn how to move data around in a context-relevant manner. This is a misrepresentation of how transformers behave and I think you should double-check the definition before dunking on other people's works.

It's not a misinterpretation. What attention does is discover association matrices which bind locations in the context window, and these associations are context sensitive. But binding locations through an association matrix is an implementation of the concept of routing, which is just moving data. Also, the link I gave regarding induction heads is explicitly moving data in the context window forward.

I consider aggregate routing to be distinct from moving data. If the context is temporary then the "data" (weights and tokenizer) stays in place. LLMs are static, they do not move data so much as they infer from it.

Re: LLMs don't do formal reasoning

#124

Post by rahimnathwani »

gitaarik wrote: post #119
rahimnathwani wrote: post #41

The paper (published 4 days ago) has this on page 10, and says that o1-mini failed to solve it correctly: Oliver picks 44 kiwis on Friday. Then he picks 58 kiwis on Saturday. On Sunday, he picks double the number of kiwis he did on Friday, but five of them were a bit smaller than average. How many kiwis does Oliver have? I pasted it into ChatGPT and Claude, and all four models I tried gave the correct answer: 4o mini…

Isn't it because this test has since been spread on the internet and the LLM's picked up on that so now they give the correct answer? Maybe try a new unique logical question. And not the same question with a few words changed, because that might still match close to data the LLM already scanned.

  the LLM's picked up on that so now they give the correct answer
The models don't just 'pick up' information that appears on the internet. They must be retrained with that new data in the training set.

I tested the models 4 days after the paper was published.

The models are retrained every few months, and the process takes much more than 4 days.

Re: LLMs don't do formal reasoning

#125

Post by YuukiRey »

I wanted to try this with Chat GPT

I buy 102 mandarins on Monday and then on Tuesday I buy another 48. On Wednesday I buy 98 apples. I didn't like the last 3 mandarins I bought. How many mandarins do I have?

You bought 102 mandarins on Monday and 48 more on Tuesday, which gives a total of:

102 + 48 = 150 mandarins.

Since you didn't like the last 3 mandarins, you subtract them:

150 - 3 = 147 mandarins.

So, you have 147 mandarins.

Re: LLMs don't do formal reasoning

#126

Post by samweb3 »

But...when I test the example, I get the right answer from 4o. Seems like they can just extend the model to identify irrelevant information over time and get the correct results more generally for similar models. -------------------------------- Let's break it down:

Friday: Oliver picks 44 kiwis. Saturday: Oliver picks 58 kiwis. Sunday: He picks double the amount he picked on Friday: 44x2=88 44×2=88 kiwis. Now, we sum all the kiwis: 44+58+88=190 Since the size of five kiwis on Sunday doesn’t affect the total count, Oliver still has: 190 kiwis.

Re: LLMs don't do formal reasoning

#127

Post by philbin »

glomgril wrote: post #59
whiplash451 wrote: post #21

I am not sure who the target audience of Gary Marcus is. Those who know about LLMs are aware that they do not reason, but also know it not very useful to repeat it over and over again and focus on other aspects of research. Those who don't know about LLMs simply learn to use them in a way that's useful in their life.

He is coming from the perspective of a long-running debate on symbolic versus statistical/data-driven approaches to modeling language structure and use. It seems in recent years he has had trouble coming to terms with the fact that at least for real-world applications of language technology, the statistical approach has simply won the war (or at worst, forms the core foundation on top of which symbolic approaches can…

[dead]
Post reply on HN