Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Can LLMs Reason and Plan?

cacm.acm.org

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Can LLMs Reason and Plan?

#31

Post by reaperman »

jbergens wrote: post #27
jasfi wrote: post #13

Earlier quoted context omitted.

What about an LLM writing an algorithm than can play Sudoku? We build things that we ourselves can't do all the time. Edit: I updated my sentence to "an LLM writing an algorithm" to make myself clearer. After reading my own sentence I realized it wasn't, sorry!

Did you read the article? The main issue with your idea is that an LLM won't know if the algorithm it created is any good, or even if it works at all. If it can't check that it will never know and never get better. You could ask it to generate a number of algorithms and then yourself choose the best one but then you have worked as a team, the LLM did not plan anything.

LLM's can integrate with a sandbox to deploy and test their code and iterate on a solution until it appears to be valid. They can also integrate with web search to go out and find presumably-valid sudoku puzzles to use for test cases if they're (likely) unable to generate valid sudoku puzzles themselves. I know it's expanding the definition of "LLM" to include a sandbox or web search, but I think it's fair because it's a reasonably practical and obvious application environment for an LLM which you plan to ask to do things like this, and I think LLMs with both these integrations will be commonplace in the next 1-2 years.

No, I don't think LLM's can "reason and plan". But I do think they can effectively mimic (fake) "reasoning and planning" and still arrive at the same result that actual reasoning and planning would yield, for reasonably common and problems of greater than trivial complexity but less than moderate complexity.

I think pretty much all of our production AI models today are limited by their lack of ability to self-assess and "goal-seek" and mutate themselves themselves to "excel". I'm not 100% sure what this would look like but I can be sure they don't have any real "drive to excel beyond". Perhaps improvements in Reinforcement Learning will uncover something like this, but I think there may need to be a paradigm shift before we invent something like that.

Re: Can LLMs Reason and Plan?

#32

Post by famouswaffles »

haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Yes LLMs can solve sudoku https://arxiv.org/abs/2305.08291

Re: Can LLMs Reason and Plan?

#33

Post by Buttons840 »

greydius wrote: post #9
greenbit wrote: post #2

These things are just statistical language models, aren't they? To the extent that when people reason or plan and then verbalize those plans, they tend to leave certain trajectories through language space, a statistical model could presumably reproduce similar, plausible sounding narratives. Doesn't mean that any kind of agency ever actually thought about those plans though, in the way that a person would.

"plausible sounding" is probably the best two word summary of LLM output. A good one word summary might be "bullshit".

I relied on GPT4 to learn linear algebra. I had a text book and GPT4. One difference is the text book was sometimes wrong (and GPT4 is the one that spotted the error in the book). If it is "bullshit", it's very helpful bullshit.

When I ask GPT4 to write some code to perform a specific task, and it does so, and the code works and correctly performs the task, is that bullshit?

It's true that LLMs are not reliable enough to trust blindly, the same applies to humans.

Re: Can LLMs Reason and Plan?

#35

Post by Buttons840 »

haltist wrote: post #3

LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.

Do 4 year old children plan? They can't solve Sudoku, so arguably not.

Re: Can LLMs Reason and Plan?

#36

Post by Communitivity »

"Plan" means a lot of things.

There has been some research in applying GPT-4 to Hierarchical Task Networks (HTN), one means of doing computerized semi-automated/automated planning of a complex task as a tree of less and less complex tasks [1].

There are other types of planning. Automated planning works better as there are more defined the tasks in a plan, less ambiguity in dependencies, more separate between the tasks. The OP article touches on that, noting LLMs are good at extracting planning knowledge but not good in their experience at creating executable plans. This is why I think the hybrid approach is best, using an LLM to inform and tweak other planning tools in order to create an executable plan.

[1] https://github.com/DaemonIB/GPT-HTN-Planner

Re: Can LLMs Reason and Plan?

#37

Post by fnordpiglet »

The obsession with whether LLMs can out perform classical AI, algorithms, solvers, and other optimizers fascinates me. Intuitively of course they aren’t doing something similar to a solver. They will never play chess better than the best for purpose chess playing system. They will never reason better than a classical reasoning system. That misses the point entirely.

They are a fascinating augmentation to existing capabilities that fills an ability to operate and do some approximation to reason and plan in an abstract semantic space, using natural language in a semantic way, to perform an abductive style of “reasoning” that has alluded us to date. Need to play chess? Use a chess playing system. But you can glue together special purpose systems for planning, reasoning, optimizing, calculating, etc, with an LLM in ways that are much more flexible and adaptable in a real world context than we’ve ever been able to achieve. That’s the magic of them.

The fact they can do pretty alright at some of these tasks is interesting and shows how powerful language is that embedded in its semantic structure is an awful lot of what you need to reason, plan, etc. But beyond the academic question, why would you not just use a provably optimal algorithm for a specific task? Interestingly, given a set of operations in context, LLMs are pretty good at identifying a situation where an operation is applicable and making the API call, and as we learn how to embed them, they’ll be able to defer to and be deferred to more accurately.

Re: Can LLMs Reason and Plan?

#38

Post by Drakim »

intended wrote: post #24
Drakim wrote: post #16

Earlier quoted context omitted.

I played asci Tic-Tac-Toe with ChatGPT using new rules that I came up with, and it was able to play using the new rules and even explained it's reasoning for the moves when I asked. I have a hard time understanding that as not including thinking and planning.

This is the call out though - the difference between a proof of concept, and then seeing what happens in production. Production and scale definitively shows that LLMs don't reason. The output is highly unpredictable. Small changes can result in absolutely unrelated outcomes. If you were to classify text using chat gpt - chat gpt switches from classification to text generation if your text is longer than a certain len…

When you poll humans for opinions, your wording can often have a very high impact on the opinions presented by folks.

Re: Can LLMs Reason and Plan?

#39

Post by Difwif »

fnordpiglet wrote: post #37

The obsession with whether LLMs can out perform classical AI, algorithms, solvers, and other optimizers fascinates me. Intuitively of course they aren’t doing something similar to a solver. They will never play chess better than the best for purpose chess playing system. They will never reason better than a classical reasoning system. That misses the point entirely. They are a fascinating augmentation to existing cap…

People are interested because of the possibility of an architecture for AGI. A system capable of generalizing to all types of problems and also being able to leverage everything you said as a tool (or even build the tool) is extremely valuable.

We're not even close to this result yet but LLMs seem the closest.

Re: Can LLMs Reason and Plan?

#40

Post by xkcd1963 »

ryanklee wrote: post #21
epups wrote: post #8

The article itself is very assertive and makes a lot of generalizations, but if you look at the source of their claims [1] you see that in the first study they are using GPT-3.5 and achieve only a 5% score on a reasoning test that largely relies on spatial intuition - some boxes need to be stacked and unstacked sequentially in a convoluted task. Then, they get criticised so they come up with another paper in which th…

> would also have welcomed a comparison to humans. Much of the criticism and skepticism around LLMs rests on a double-standard that itself rests on an almost embarrassing lack of understanding of how humans themselves operate.

Absolutely!
Post reply on HN