Live data from Hacker News

LLMs don't do formal reasoning

garymarcus.substack.com

21–30 of 128 posts

Re: LLMs don't do formal reasoning

#21
I am not sure who the target audience of Gary Marcus is.

Those who know about LLMs are aware that they do not reason, but also know it not very useful to repeat it over and over again and focus on other aspects of research.

Those who don't know about LLMs simply learn to use them in a way that's useful in their life.

Re: LLMs don't do formal reasoning

#22
So it means that throwing data and computing into LLMs won't make them intelligent as opposed many people claiming that will happen. Also they don't play well with the situations that are not in the dataset since they are just extrapolating from what they are trained without any real understanding.

Re: LLMs don't do formal reasoning

#23

One of the things that kinda illustrate this for me, is that an LLM always uses the same time to process a prompt of the same length. No matter how complicated the problem is. Obviously the complexity of the problem is not actually taken into account.

the o1 model definitely has a somewhat big variance in how long the task takes depending on what you ask it to do

Re: LLMs don't do formal reasoning

#24
post #5

This trope of proclaiming some critical flaw in the functioning of LLMs with the implication that they therefore should not be used is getting boring. LLMs are far from perfect but they can be a very useful tool that, used well, can add significant value in spite of their flaws. Large numbers of people and businesses are extracting huge value from the use of LLMs every single day. Some people are building what will b…

It's probably the other way around actually.

The average person assumes LLMs are intelligent and all this AI thing will end up replacing them. This has created a distorted perception of the tech which has had multiple consequences. It's necessary to change this perception so that it better adjusts with reality.

Re: LLMs don't do formal reasoning

#25

One of the things that kinda illustrate this for me, is that an LLM always uses the same time to process a prompt of the same length. No matter how complicated the problem is. Obviously the complexity of the problem is not actually taken into account.

This is only true if the output is the same length (which should be exceptionally rare if the input text is different).

Re: LLMs don't do formal reasoning

#26

One of the things that kinda illustrate this for me, is that an LLM always uses the same time to process a prompt of the same length. No matter how complicated the problem is. Obviously the complexity of the problem is not actually taken into account.

Wait what ? Is that real?

Yes. OpenAI's o1 model is an attempt to address this, by letting the model choose to "think" by generating hidden tokens for a variable amount of time before producing the visible output tokens. But each token whether hidden or visible still takes a fixed amount of compute.

Re: LLMs don't do formal reasoning

#27
post #11
post #7

Earlier quoted context omitted.

Are you suggesting Humans can't do formal reasoning? Because you can easily teach a four year old not to make illegal moves in chess with very little instructions, and by 10 geniuses like Terence Tao were discussing open math problems with Erdos. If anything this article adds further evidence that whatever the architecture of the human brain it is very different to an LLM architecture.

> Are you suggesting Humans can't do formal reasoning? most of them can't. they actively vote against their own interests and everybody else around them. if you confront them with facts and figures, they ignore it and resort to emotional appeals. just because you can find one child that can play chess at age 4 doesn't mean that the rest won't just eat the pieces and shit on the board.

I've never been a fan of the argument of how voting against your own interest is some gap in logic. To me it can mean multiple things, one could be not understanding the implications of what that vote could mean, the other is a full understanding but wanting to put "the greater good" above the self. I would not want to live in a society where everyone only votes for their own interests.

Re: LLMs don't do formal reasoning

#28
ChatGPT o1-preview was not flummoxed by the small kiwis, and even called out the extraneous detail:

```

To determine the total number of kiwis Oliver has, we’ll sum up the kiwis he picked on each day:

1. Friday: Oliver picks 44 kiwis.

2. Saturday: He picks 58 kiwis.

3. Sunday: He picks double the number he did on Friday, so 2 × 44 = 88 kiwis.

Adding them up:

44 (Friday) + 58 (Saturday) + 88 (Sunday) = 190 kiwis

The mention of five smaller-than-average kiwis on Sunday doesn’t affect the total count unless specified otherwise.

Answer: 190

```

Re: LLMs don't do formal reasoning

#29

One of the things that kinda illustrate this for me, is that an LLM always uses the same time to process a prompt of the same length. No matter how complicated the problem is. Obviously the complexity of the problem is not actually taken into account.

Wait what ? Is that real?

Yes, you never tried it? I always get the same tokens/s from my local LLM setup no matter what I put in (and because it's local there are no hidden resources the cloud might have added to solve my extra-hard problem).

It does depend on the context + prompt length but for those the results are pretty static. It's clear to me that an LLM doesn't actually reason. Which is not something it's really been built to do so I'm not sure if it's a bad thing. The problem is more that people expect it to do that. Probably because it sounds so human so they ascribe human-like skills to it.

Re: LLMs don't do formal reasoning

#30
Tangential but I do wonder how much it's actually going to cost to use these systems once the investor money gets turned off and they want a return on investment. Given that the systems are only getting bigger it can't be cheap.
Post reply on HN