Live data from Hacker News

LLMs don't do formal reasoning

garymarcus.substack.com

31–40 of 128 posts

Re: LLMs don't do formal reasoning

#31
post #5

This trope of proclaiming some critical flaw in the functioning of LLMs with the implication that they therefore should not be used is getting boring. LLMs are far from perfect but they can be a very useful tool that, used well, can add significant value in spite of their flaws. Large numbers of people and businesses are extracting huge value from the use of LLMs every single day. Some people are building what will b…

> Yet in the face of this we still see a population of naysayers who appear intent on rubbishing LLMs at any cost.

What was the cost in this case ? It's just an experiment and I think your reaction is way too emotional for some reason.

Re: LLMs don't do formal reasoning

#32
post #25

One of the things that kinda illustrate this for me, is that an LLM always uses the same time to process a prompt of the same length. No matter how complicated the problem is. Obviously the complexity of the problem is not actually taken into account.

This is only true if the output is the same length (which should be exceptionally rare if the input text is different).

That's true, I was talking about tokens/sec output but I should have specified.

Re: LLMs don't do formal reasoning

#33

One of the things that kinda illustrate this for me, is that an LLM always uses the same time to process a prompt of the same length. No matter how complicated the problem is. Obviously the complexity of the problem is not actually taken into account.

Wait what ? Is that real?

Yes. In the end, LLMs are a sequence of matrix multiplications and since they don't loop internally, every output token gets the same number of internal processing steps, no matter what the input is. Only the input length is relevant because some steps can be skipped if the input buffer is not full.

Re: LLMs don't do formal reasoning

#34

They’re arguing since it’s not close to perfect, it’s not useful? Seems like a straw man.

I don't see anybody arguing that it isn't useful in general, just that it's unreliable, and that we need to change or add to the fundamental architecture to make progress.

Re: LLMs don't do formal reasoning

#35
post #24
post #5

This trope of proclaiming some critical flaw in the functioning of LLMs with the implication that they therefore should not be used is getting boring. LLMs are far from perfect but they can be a very useful tool that, used well, can add significant value in spite of their flaws. Large numbers of people and businesses are extracting huge value from the use of LLMs every single day. Some people are building what will b…

It's probably the other way around actually. The average person assumes LLMs are intelligent and all this AI thing will end up replacing them. This has created a distorted perception of the tech which has had multiple consequences. It's necessary to change this perception so that it better adjusts with reality.

You are unlikely to reach the average person by posting an analysis of GSM-NoOp on substack.

Re: LLMs don't do formal reasoning

#36

Tangential but I do wonder how much it's actually going to cost to use these systems once the investor money gets turned off and they want a return on investment. Given that the systems are only getting bigger it can't be cheap.

If it is a bubble that pops, it has a high cost of goods sold, which could lead to a big burst!

Re: LLMs don't do formal reasoning

#37
post #5

This trope of proclaiming some critical flaw in the functioning of LLMs with the implication that they therefore should not be used is getting boring. LLMs are far from perfect but they can be a very useful tool that, used well, can add significant value in spite of their flaws. Large numbers of people and businesses are extracting huge value from the use of LLMs every single day. Some people are building what will b…

To be fair, and in case it isn’t obvious: this is kinda this guy’s whole schtick. And has been for decades:

  The inability of standard neural network architectures to reliably extrapolate — and reason formally — has been the central theme of my own work back to 1998 and 2001, and has been a theme in all of my challenges to deep learning, going back to 2012, and LLMs in 2019.
Basically he sees his role in human development as a Diogenes-esque figure, a cynic whose job is to loudly and frequently point out flaws in the rising tide of connectionist AI research — to throw a plucked chicken at Socrates to disprove his description of humans as featherless bipeds, so to speak. Except now, for better or worse, the poultry-tossing has been replaced by polemics on Twitter and Substack.

The point isn’t to contribute to expert-level discourse with incremental clarifications (like most academics do), but rather to keep the overall zeitgeist around the technology in check. I absolutely agree that he’s not a useful figure for engineers trying to employ the tools available to them; I think his audience is more like “voters” or “university donors” or “department heads” — in other words, people fretting over long term directions.

When he started connectionism was the underdog camp, and he’s lived to see it take over AI to such an extreme extent that most laypeople would honestly say that AI didn’t exist until, like, 5 years ago. I think we can all relate to how frustrating that must feel!

Plus he’s fun. He’s not quite at guru levels of dishonesty, but he’s still got that guru flair for the dramatic. He’s worth a substack sub just to get the flip side of every big event, IMO!

Re: LLMs don't do formal reasoning

#38
post #7
post #4

You could substitute "LLMs" -> "Humans" and the statement would also be true.

Are you suggesting Humans can't do formal reasoning? Because you can easily teach a four year old not to make illegal moves in chess with very little instructions, and by 10 geniuses like Terence Tao were discussing open math problems with Erdos. If anything this article adds further evidence that whatever the architecture of the human brain it is very different to an LLM architecture.

No, but that's irrelevant to whether humans do formal reasoning. Generally, we don't. Eg. https://m.youtube.com/watch?v=UBVV8pch1dM&pp=ygUhdGhpbmtpbmc...

Re: LLMs don't do formal reasoning

#39

Getting tired of seeing this guy's bad arguments get signal boosted. I posted this comment on another LLM thread on the front page today, and I'll just repost it here: LLMs aren't totally out of scope of mathematical reasoning. LLMs roughly do two things, move data around, and recognize patterns. Reasoning leans heavily on moving data around according to context-sensitive rules. This is well within the scope of LLMs.…

That's an interesting rebuttal if you can suggest near-future architectures which don't require their own nuclear power plants to reliably calculate 13 x 54.

Re: LLMs don't do formal reasoning

#40
The thing is, from a written human readable text, there is no single formal reasoning. The text itself is not formal. The facts that kiwis are bigger or smaller might seem irrelevant for counting the amount of kiwis, but there is no formal proof of that possible. I might argue that counting might include volume or weight, you might argue that one kiwi is one kiwi. So saying that llm's don't do formal reasoning is not saying anything, as it doesn't mean anything when you start from written sentences.

My point being, LLMs are capable of reasoning and formal reasoning is meaningless in the context.

Post reply on HN