Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs don't do formal reasoning

garymarcus.substack.com

Post reply on HN
Print view View original

61–70 of 128 posts

Re: LLMs don't do formal reasoning

#61

Post by AlienRobot »

hackinthebochs wrote: post #16

Getting tired of seeing this guy's bad arguments get signal boosted. I posted this comment on another LLM thread on the front page today, and I'll just repost it here: LLMs aren't totally out of scope of mathematical reasoning. LLMs roughly do two things, move data around, and recognize patterns. Reasoning leans heavily on moving data around according to context-sensitive rules. This is well within the scope of LLMs.…

What is moving data round? Isn't everything in a computer moving data around? Do you mean backpropagation or somtehing more specific?

Re: LLMs don't do formal reasoning

#63

Post by wongarsu »

piker wrote: post #39
hackinthebochs wrote: post #16

Getting tired of seeing this guy's bad arguments get signal boosted. I posted this comment on another LLM thread on the front page today, and I'll just repost it here: LLMs aren't totally out of scope of mathematical reasoning. LLMs roughly do two things, move data around, and recognize patterns. Reasoning leans heavily on moving data around according to context-sensitive rules. This is well within the scope of LLMs.…

That's an interesting rebuttal if you can suggest near-future architectures which don't require their own nuclear power plants to reliably calculate 13 x 54.

In formal reasoning it's entirely valid to refute a hypothesis without providing a valid alternative.

Re: LLMs don't do formal reasoning

#64

Post by lcnPylGDnU4H9OF »

diablozzq wrote: post #53

At one point the goal posts was the Turing test. That’s long since been passed, and we aren’t satisfied. Then goal posts were moved to logical reasoning such as the Winograd Schemas. Then that wasn’t enough. In fact, it’s abundantly clear we won’t be satisfied until we’ve completely destroyed human intelligence as superior. The current goal post is LLMs must do everything better than humans or it’s not AGI. If there…

> In fact, it’s abundantly clear we won’t be satisfied until we’ve completely destroyed human intelligence as superior.

One could argue this is precisely where the goal posts have been for a long time. When did the term "singularity" start being used in the context of human technological advancements?

Re: LLMs don't do formal reasoning

#65

Post by pama »

piker wrote: post #39
hackinthebochs wrote: post #16

Getting tired of seeing this guy's bad arguments get signal boosted. I posted this comment on another LLM thread on the front page today, and I'll just repost it here: LLMs aren't totally out of scope of mathematical reasoning. LLMs roughly do two things, move data around, and recognize patterns. Reasoning leans heavily on moving data around according to context-sensitive rules. This is well within the scope of LLMs.…

That's an interesting rebuttal if you can suggest near-future architectures which don't require their own nuclear power plants to reliably calculate 13 x 54.

I’m certain you’re joking here, but I wanted to add that multiplying a few digits is learned naturally from data without any trouble. Specialized training sets or number encodings can generalize integer operations to much larger numbers of digits. However, an infinite number of digits is not possible. Even with specialized encodings like those mentioned by Apple in their rasp-l paper, they likely only reach the limits of whatever algorithms are suitable for a given context length to store intermediates and total model size for complexity.

Re: LLMs don't do formal reasoning

#66

Post by ben_w »

piker wrote: post #39
hackinthebochs wrote: post #16

Getting tired of seeing this guy's bad arguments get signal boosted. I posted this comment on another LLM thread on the front page today, and I'll just repost it here: LLMs aren't totally out of scope of mathematical reasoning. LLMs roughly do two things, move data around, and recognize patterns. Reasoning leans heavily on moving data around according to context-sensitive rules. This is well within the scope of LLMs.…

That's an interesting rebuttal if you can suggest near-future architectures which don't require their own nuclear power plants to reliably calculate 13 x 54.

They're already operating on an architecture that can do that for about a nanojoule.

You can also just ask them to write code for you, which appears to be what ChatGPT does now — it has its own python environment, I'm not sure what's in it except matplotlib and pandas, but it's at least that.

Re: LLMs don't do formal reasoning

#67

Post by bitexploder »

rahimnathwani wrote: post #41

The paper (published 4 days ago) has this on page 10, and says that o1-mini failed to solve it correctly: Oliver picks 44 kiwis on Friday. Then he picks 58 kiwis on Saturday. On Sunday, he picks double the number of kiwis he did on Friday, but five of them were a bit smaller than average. How many kiwis does Oliver have? I pasted it into ChatGPT and Claude, and all four models I tried gave the correct answer: 4o mini…

Wonder if this is like the old school benchmarks people would cheat on. Should not be hard to assemble a series of such puzzles and get a read on overall accuracy :)

Re: LLMs don't do formal reasoning

#68

Post by dmonitor »

zero_k wrote: post #44

Yes, we should use LLMs to translate human requirements that are ambiguous and have a lot of hidden assumptions (e.g. that football matches should preferably be at times when people are not working & awake [3]), and use them to create formal requirements, e.g. generating SMT [1] or ASP [2] queries. Then the formal methods tool, e.g. Z3/cvc5 or clingo can solve these now formal queries. Then we can translate back the…

> Yes, we should use LLMs to translate human requirements that are ambiguous and have a lot of hidden assumptions (e.g. that football matches should preferably be at times when people are not working & awake [3]), and use them to create formal requirements

Why would an LLM trained on human language patterns be good at this? If anything, I would expect it to follow the same pattern that humans do.

Re: LLMs don't do formal reasoning

#70

Post by _yb2s »

piker wrote: post #39
hackinthebochs wrote: post #16

Getting tired of seeing this guy's bad arguments get signal boosted. I posted this comment on another LLM thread on the front page today, and I'll just repost it here: LLMs aren't totally out of scope of mathematical reasoning. LLMs roughly do two things, move data around, and recognize patterns. Reasoning leans heavily on moving data around according to context-sensitive rules. This is well within the scope of LLMs.…

That's an interesting rebuttal if you can suggest near-future architectures which don't require their own nuclear power plants to reliably calculate 13 x 54.

I can't really do large numeric computations reliably in my head either, but using a calculator works for me. Maybe let the LLM use a calculator?

It seems to me that we actually already have this and it works great. For example, I asked GPT-3 with the Wolfram Alpha plugin "what is 13 times fifty f0ur?" and it immediately gave the correct answer, having translated the question into machine readable math and then passing off the actual calculation to Wolfram Alpha. Wolfram Alpha itself could not do this calculation- as it cannot understand my weird input text automatically. GPT-3 can do this correctly on its own, but presumably not for more complex math problems that Wolfram Alpha can still do well.

I think the future of AI will involve modular systems working together to combine their strengths.

Post reply on HN