Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs Reason and Plan?

cacm.acm.org

Post reply on HN
Print view View original

41–50 of 51 posts

Re: Can LLMs Reason and Plan?

#41

Post by kat_rebelo »

LLM's only work on the data they have been trained on so all outputs are merely based on information that has already been written about by a human. Furthermore, LLM's do not truly "understand" even first order causal relationships, meaning whatever it plans will have no foresight to evaluate how a plan it generates will impact downstream components of a complex system.

LLM's live in "the world that has been written about", not the real world, and thus cannot formulate new ideas or hypothesis other than by accident. This, coupled with the lack of an ontological system for evaluating the validity of the statements it makes about a complex system, and, its lack of causal reasoning, means they cannot effectively plan.

I've worked on research related to causality that used LLM's (admittedly, pre ChatGPT and using much smaller models) and it was not uncommon to see extremely bogus causal relationships inferred such as "rising cost of living in NYC caused a flood in Argentina".

Re: Can LLMs Reason and Plan?

#42

Post by fnordpiglet »

Difwif wrote: post #39
fnordpiglet wrote: post #37

The obsession with whether LLMs can out perform classical AI, algorithms, solvers, and other optimizers fascinates me. Intuitively of course they aren’t doing something similar to a solver. They will never play chess better than the best for purpose chess playing system. They will never reason better than a classical reasoning system. That misses the point entirely. They are a fascinating augmentation to existing cap…

People are interested because of the possibility of an architecture for AGI. A system capable of generalizing to all types of problems and also being able to leverage everything you said as a tool (or even build the tool) is extremely valuable. We're not even close to this result yet but LLMs seem the closest.

I agree, but I think there’s an overly large emphasis on picking at LLMs (in)ability to reason in a structured way independent of its semantic language expectation model. It’s usually used as a “see LLM sucks” while the other side holds to “see it can do everything.” Neither is right. LLMs are amazing, but they can’t do everything, even if they can seem to do many things most of the time. While interesting to investigate the boundaries, I think the passion should be more directed towards “how do we wire the piece together and for what and where”

Re: Can LLMs Reason and Plan?

#43

Post by gamblor956 »

famouswaffles wrote: post #32
haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Yes LLMs can solve sudoku https://arxiv.org/abs/2305.08291

No, the LLM did not solve sudoku. They simply linked a sudoku solver to an LLM. All the LLM did was constantly invoke the solver. The solver can be downloaded from github, and works independently of the LLM.

All that experiment demonstrated is that you can use an LLM to run through a state tree...which is something that simple machine agents have been able to do for a few decades.

Re: Can LLMs Reason and Plan?

#44

Post by circuit10 »

appplication wrote: post #28
circuit10 wrote: post #26

Earlier quoted context omitted.

Would an average human be able to do that on a strict time limit (which LLMs effectively have as they do a fixed amount of computation per token)?

The mark for reasoning in a computer program is not whether or not a human can do so on the same time limit. A calculator doesn’t reason but solves 658236 x 37854285 faster than anything human could. You could give an LLM days per token and it wouldn’t change its capabilities regarding reasoning.

I'm not saying that this is proof that it can reason, but rather that the fact that it can't isn't proof that it can't on its own

Re: Can LLMs Reason and Plan?

#45

Post by theonlybutlet »

kat_rebelo wrote: post #41

LLM's only work on the data they have been trained on so all outputs are merely based on information that has already been written about by a human. Furthermore, LLM's do not truly "understand" even first order causal relationships, meaning whatever it plans will have no foresight to evaluate how a plan it generates will impact downstream components of a complex system. LLM's live in "the world that has been written…

Neither can a human. Unless the human subconsciously re-evaluates their output and refines it before speaking, much like running the LLM output through the model again. Or if they've learnt it through past experience reinforcing that pathway much like LLM learning and adjusting weights to factor that in so it would impact its output in future.

Re: Can LLMs Reason and Plan?

#46

Post by theonlybutlet »

haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Of course it can given the correct framework. I think that's more a limitation of the structure of its interface and programming around it. Allow it to consider and iterate on its own output and it'll get there.

Re: Can LLMs Reason and Plan?

#47

Post by famouswaffles »

gamblor956 wrote: post #43
famouswaffles wrote: post #32

Earlier quoted context omitted.

Yes LLMs can solve sudoku https://arxiv.org/abs/2305.08291

No, the LLM did not solve sudoku. They simply linked a sudoku solver to an LLM. All the LLM did was constantly invoke the solver. The solver can be downloaded from github, and works independently of the LLM. All that experiment demonstrated is that you can use an LLM to run through a state tree...which is something that simple machine agents have been able to do for a few decades.

>They simply linked a sudoku solver to an LLM. All the LLM did was constantly invoke the solver.

No they didn't just link a solver.

The LLM does the filling. It makes an attempt, the checker checks if it's a valid move for sudoku, If not then returns to the previous state(node) and so on until solved. The history of attempts is stored and retrieved every time the LLM backtracks

Re: Can LLMs Reason and Plan?

#49

Post by haukurb »

famouswaffles wrote: post #29

A test that doesn't properly inform you someone/thing's abilities is a rubbish test. From their paper, you would think being able to pilot robots to handle chores or stack objects or whatever would be well beyond LLMs. But it's not. https://tidybot.cs.princeton.edu/ https://innermonologue.github.io/ Anyone who wants LLMs to plan and is actually interested in teasing the extent of those abilities knows how to structur…

Their example prompt is so bad that I'm split between them being wildly incompetent at understanding how transformers or autoregressiveness works (and general LLM dynamics) or they are deliberately obfuscating the task and prompt representation in bad faith.

If one really wants to know if LLMs "are capable of planning", one should keep an open mind for how planning behavior can manifest in textual form, then actually try to find any manifestations. Imposing one's view of how planning should look like is bad science. All they've proved is that their task format sucks for current generation LLMs.

Re: Can LLMs Reason and Plan?

#50

Post by haltist »

Buttons840 wrote: post #35
haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Do 4 year old children plan? They can't solve Sudoku, so arguably not.

Children have no explict conceptions of numbers and logic so they obviously do not plan the same way someone with explicit knowledge and understanding of logic reasons and plans. One could argue they have implicit understanding as members of a species known for inventing mathematics but that's more of a philosophical argument than a scientific one.
Post reply on HN