[deleted]
Can LLMs Reason and Plan?
11–20 of 51 posts
Re: Can LLMs Reason and Plan?
#12LLMs can write a plan, and they can write about their reasoning. The question is, are they good enough?
We may need another generation of LLMs that are more competent.
Re: Can LLMs Reason and Plan?
#13LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.
What about an LLM writing an algorithm than can play Sudoku?
We build things that we ourselves can't do all the time.
Edit: I updated my sentence to "an LLM writing an algorithm" to make myself clearer. After reading my own sentence I realized it wasn't, sorry!
Re: Can LLMs Reason and Plan?
#14LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.
What about an LLM writing an algorithm than can play Sudoku? We build things that we ourselves can't do all the time. Edit: I updated my sentence to "an LLM writing an algorithm" to make myself clearer. After reading my own sentence I realized it wasn't, sorry!
Of course we can write an algorithm to solve Sudoku. The commenter specifically is talking about LLMs, not algorithms in general.
Re: Can LLMs Reason and Plan?
#15Earlier quoted context omitted.
What about an LLM writing an algorithm than can play Sudoku? We build things that we ourselves can't do all the time. Edit: I updated my sentence to "an LLM writing an algorithm" to make myself clearer. After reading my own sentence I realized it wasn't, sorry!
Of course we can write an algorithm to solve Sudoku. The commenter specifically is talking about LLMs, not algorithms in general.
I'm making the point that they're not as limited as they seem at first glance.
Re: Can LLMs Reason and Plan?
#16LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.
I played asci Tic-Tac-Toe with ChatGPT using new rules that I came up with, and it was able to play using the new rules and even explained it's reasoning for the moves when I asked. I have a hard time understanding that as not including thinking and planning.
Re: Can LLMs Reason and Plan?
#17Earlier quoted context omitted.
I would hope the next generation of softwares engineers are not the kind to blindly buy into media hype.
The next generation (going by some comments I’ve read on HN) are already non-programmers gluing (I assume terrible) LLM-generated code together until it (appears like it) works. Hell, my non-programmer brother recently sent me a message like “can you think of a way to fix this script, ChatGPT isn’t managing” (sends me a badly written AI-generated script)
> “can you think of a way to fix this script, ChatGPT isn’t managing”
haha, who's the glue coder now!
edit: I chat with a 13 year old glue coder one time. He couldn't really code at all but he could find the components and was excellent at prompting people for help. He showed many things like map api's interacting with many other things. How long did that take you to put together? uhh 2 hours. My mind was blown.
Re: Can LLMs Reason and Plan?
#18LLMs can not plan. There is no LLM that can solve sudoku puzzles by executing the obvious constraint propagation algorithm with backtracking. Therefore, LLMs can neither reason nor plan. Software is not magic and the fact that a lot of people are starting to think that it is should be concerning for the folks training the next generation of software engineers.
I played asci Tic-Tac-Toe with ChatGPT using new rules that I came up with, and it was able to play using the new rules and even explained it's reasoning for the moves when I asked. I have a hard time understanding that as not including thinking and planning.
Sounds interesting! Can you please share the transcript?
Re: Can LLMs Reason and Plan?
#19These things are just statistical language models, aren't they? To the extent that when people reason or plan and then verbalize those plans, they tend to leave certain trajectories through language space, a statistical model could presumably reproduce similar, plausible sounding narratives. Doesn't mean that any kind of agency ever actually thought about those plans though, in the way that a person would.
> To the extent that when people reason or plan and then verbalize those plans
This is a large and dubious assumption. Most of what humans do require step coordination, but almost none of it is a result of explicit planning or verbalization.
Re: Can LLMs Reason and Plan?
#20Earlier quoted context omitted.
I played asci Tic-Tac-Toe with ChatGPT using new rules that I came up with, and it was able to play using the new rules and even explained it's reasoning for the moves when I asked. I have a hard time understanding that as not including thinking and planning.
Sounds interesting! Can you please share the transcript?
I'm looking for it, but unfortunately I can't find it, which is weird because I've never deliberately deleted any of the transcripts.
The homemade rules I used were as follows (each for a separate round):
1. The first player to get three in a row loses.
2. The first player to take the center square loses.
3. Players can place their marker in squares already taken.
Admittedly it didn't nail it perfectly, a few times it messed up, but most of the time it was able to follow the rules as explained without issue, and could explain it's reasoning when I asked it why it picked that square.