You would too. You just don't speak it out loud.
ChatGPT has trouble giving an answer before explaining its reasoning
31–40 of 85 posts
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#32Earlier quoted context omitted.
Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?
Here's a thought: If you let ChatGPT be idle, thinking to itself, dreaming and planning, this might actually cross the boundary towards sentience - of what we would call somebody who is alive. So there might be some safety and moral concerns.
Why would hidingwhat it's already doing make it more sentient?
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#33People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…
Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?
This can certainly be done. Here's one example from 2021 demonstrating training an LLM to use a scratchpad ("talking to itself") to greatly improve accuracy on arithmetic problems:
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#34On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily. I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser. It's hard for me to believe that t…
ChatGPT doesn't well understand relationships between numbers. There are far too many of them, compared to words, since every slight perturbation of a number is a different valid number. (Also, I'm not sure if it even treated individua5 digits as separate tokens, but it might. Someone with API access can check.)
To give it a fair shot, you need to describe the problem using logical conceptual vocabulary, not numbers.
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#35The challenge here is that ChatGPT and other LLMs can only think out loud. They only "think" through writing, and that's always displayed to the user. Has anyone tried giving LLMs a scratchpad where the model could e.g. run the pipeline in order, generate the poem, and then explicitly publish it to the user without showing the earlier steps?
You can ask GPT what would be a result of executing a python program, for which a multiple step calculation is needed. It will readily output the result, with no thinking aloud.
Does it have a python coprocessor?
I guided it to write a program for me, which it did correctly, and then I asked to evaluate it on different numeric inputs. It got correct answers for small numbers and the first few positions of map(thefunction,[1,2,3,4,5,6,7,8,9]) before wandering off into bad fuesses.
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#36We did some work in exploring why spelling out the rationale before the answer works so well!
Talk: https://madaan.github.io/res/presentations/TwoToTango.pdf
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#37People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…
Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?
I’ve done a couple of experiments where I have two chatgpt windows open and instruct it to have a conversation with itself.
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#38Earlier quoted context omitted.
Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?
Came back to this post after thinking for a while precisely to mention this. Right now, ChatGPT is sort of forced to "think" and talk at the same time, so it's hard for it to "reason" ahead of answering. But, if we allowed him to produce some tokens in silence prior to answering, perhaps it could give even better answers.
It's fun to watch these techniques slowly evolve into something resembling regular old human thought
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#39People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…
You can really screw with this by asking it to output responses in upside-reverse order homographs.. Then it has to start with the last word and work backwards.
Re: ChatGPT has trouble giving an answer before explaining its reasoning
#40Earlier quoted context omitted.
Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?
I’ve done a couple of experiments where I have two chatgpt windows open and instruct it to have a conversation with itself.
Did something interesting happen?