Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs Reason and Plan?

cacm.acm.org

Post reply on HN
Print view View original

21–30 of 51 posts

Re: Can LLMs Reason and Plan?

#21

Post by ryanklee »

epups wrote: post #8

The article itself is very assertive and makes a lot of generalizations, but if you look at the source of their claims [1] you see that in the first study they are using GPT-3.5 and achieve only a 5% score on a reasoning test that largely relies on spatial intuition - some boxes need to be stacked and unstacked sequentially in a convoluted task. Then, they get criticised so they come up with another paper in which th…

> would also have welcomed a comparison to humans.

Much of the criticism and skepticism around LLMs rests on a double-standard that itself rests on an almost embarrassing lack of understanding of how humans themselves operate.

Re: Can LLMs Reason and Plan?

#22

Post by BoorishBears »

jasfi wrote: post #12

LLMs can write a plan, and they can write about their reasoning. The question is, are they good enough? We may need another generation of LLMs that are more competent.

Actually LLMs can write a better plan than the author:

The prompt format made absolutely no sense considering they had decided to translate away from the PDDL to arbitrary natural language. So to avoid triggering their overused Clever Hans defense, I fed GPT 4 their prompt with only the instruction:

    "Think critically about how we could represent these rules in a way that's clearer to an LLM. Lean into using coding style identifiers where possible, and JSON formatting"
(Hopefully the author won't claim telling a model to generate JSON is secretly telling it how to move blocks!)

In a fresh context window at 0 temp and gpt-4-0613 I entered the JSON formatted rules it generated, along with the instruction:

    Return a JSON array of [{[],, }] that results in goal state
... the resulting answer solved their failed few-shot example with zero-shot

_

Also funny blunder: their chain of thought example generates thoughts... after the action. Surely the author understands a transformer model can't rely on an ungenerated token to affect the action taken?

Edit: Also their "disguised difficulty" version was wrong?

> To perform Attack action, the following facts need to be true: Province object, Planet object, Harmony.

The single word "Harmony" somehow replaced replaces "Your hands are empty"?

Before that they claim it's meant to be a 1:1 replacement of entities, but the actual disguised versions are not longer valid instructions

Re: Can LLMs Reason and Plan?

#24

Post by intended »

Drakim wrote: post #16
haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

I played asci Tic-Tac-Toe with ChatGPT using new rules that I came up with, and it was able to play using the new rules and even explained it's reasoning for the moves when I asked. I have a hard time understanding that as not including thinking and planning.

This is the call out though - the difference between a proof of concept, and then seeing what happens in production.

Production and scale definitively shows that LLMs don't reason. The output is highly unpredictable. Small changes can result in absolutely unrelated outcomes.

If you were to classify text using chat gpt - chat gpt switches from classification to text generation if your text is longer than a certain length.

LLM based Agents are the place where LLM reasoning died in practice.

I really hope that there is a change, maybe larger context windows will allow the generated text to "reason". Although, that would not necessarily be reasoning.

Even the article linked,

"Indeed, LLMs make it easy to get problem-specific knowledge as long as we are willing to relax correctness requirements of that knowledge."

LLMs generate text that approximates what a plan would "sound" like. It doesn't plan.

Re: Can LLMs Reason and Plan?

#25

Post by jasfi »

BoorishBears wrote: post #22
jasfi wrote: post #12

LLMs can write a plan, and they can write about their reasoning. The question is, are they good enough? We may need another generation of LLMs that are more competent.

Actually LLMs can write a better plan than the author: The prompt format made absolutely no sense considering they had decided to translate away from the PDDL to arbitrary natural language. So to avoid triggering their overused Clever Hans defense, I fed GPT 4 their prompt with only the instruction: "Think critically about how we could represent these rules in a way that's clearer to an LLM. Lean into using coding st…

I'm not surprised, GPT-4 really is very good, you just have to prompt it well enough. Often you don't even have to be a prompt guru.

Re: Can LLMs Reason and Plan?

#26

Post by circuit10 »

haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Would an average human be able to do that on a strict time limit (which LLMs effectively have as they do a fixed amount of computation per token)?

Re: Can LLMs Reason and Plan?

#27

Post by jbergens »

jasfi wrote: post #13
haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

What about an LLM writing an algorithm than can play Sudoku? We build things that we ourselves can't do all the time. Edit: I updated my sentence to "an LLM writing an algorithm" to make myself clearer. After reading my own sentence I realized it wasn't, sorry!

Did you read the article? The main issue with your idea is that an LLM won't know if the algorithm it created is any good, or even if it works at all. If it can't check that it will never know and never get better. You could ask it to generate a number of algorithms and then yourself choose the best one but then you have worked as a team, the LLM did not plan anything.

Re: Can LLMs Reason and Plan?

#28

Post by appplication »

circuit10 wrote: post #26
haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Would an average human be able to do that on a strict time limit (which LLMs effectively have as they do a fixed amount of computation per token)?

The mark for reasoning in a computer program is not whether or not a human can do so on the same time limit. A calculator doesn’t reason but solves 658236 x 37854285 faster than anything human could.

You could give an LLM days per token and it wouldn’t change its capabilities regarding reasoning.

Re: Can LLMs Reason and Plan?

#29

Post by famouswaffles »

A test that doesn't properly inform you someone/thing's abilities is a rubbish test. From their paper, you would think being able to pilot robots to handle chores or stack objects or whatever would be well beyond LLMs. But it's not.

https://tidybot.cs.princeton.edu/ https://innermonologue.github.io/

Anyone who wants LLMs to plan and is actually interested in teasing the extent of those abilities knows how to structure planning requests.

It's extra funny because humans can't actually generate plans the way he tests LLMs to either.

Also Seeing output from GPT that demonstrates intelligence, reasoning, or whatever, and saying it is not real reasoning/Intelligence etc, is like looking at a plane soar and saying that the plane is fake flying. And this isn't a nature versus artificial thing either. The origin point is entirely arbitrary.

You could just as easily move the origin to Bees and say, "oh, birds aren't really flying". You could move it to planes and say, "oh, helicopters aren't really flying." It's a very meaningless statement.

Internal processes are entirely irrelevant.

Re: Can LLMs Reason and Plan?

#30

Post by [deleted] »

intended wrote: post #24
Drakim wrote: post #16

Earlier quoted context omitted.

I played asci Tic-Tac-Toe with ChatGPT using new rules that I came up with, and it was able to play using the new rules and even explained it's reasoning for the moves when I asked. I have a hard time understanding that as not including thinking and planning.

This is the call out though - the difference between a proof of concept, and then seeing what happens in production. Production and scale definitively shows that LLMs don't reason. The output is highly unpredictable. Small changes can result in absolutely unrelated outcomes. If you were to classify text using chat gpt - chat gpt switches from classification to text generation if your text is longer than a certain len…

[deleted]
Post reply on HN