Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT has trouble giving an answer before explaining its reasoning

blog.valentin.sh

Post reply on HN
Print view View original

21–30 of 85 posts

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#21

Post by JieJie »

swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Hear hear!

There is an awful lot of binary thinking going on by humans as we are grappling with the implications of the complex behavior going on in our computers.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#22

Post by lostmsu »

civilized wrote: post #18

On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily. I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser. It's hard for me to believe that t…

> I am continuing the conversation and I am now being explicitly told that, while 1,000,000 kg is much heavier than 22,600 kg, it doesn't change the fact that the osmium is heavier than the helium because the osmium is denser.

Oh, the nature of many Internet discussions.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#23

Post by civilized »

lostmsu wrote: post #22
civilized wrote: post #18

On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily. I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser. It's hard for me to believe that t…

> I am continuing the conversation and I am now being explicitly told that, while 1,000,000 kg is much heavier than 22,600 kg, it doesn't change the fact that the osmium is heavier than the helium because the osmium is denser. Oh, the nature of many Internet discussions.

Are we sure we can have artificial general intelligence when humans don't even have general intelligence?

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#26

Post by jtbayly »

swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#27

Post by huevosabio »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

Came back to this post after thinking for a while precisely to mention this.

Right now, ChatGPT is sort of forced to "think" and talk at the same time, so it's hard for it to "reason" ahead of answering.

But, if we allowed him to produce some tokens in silence prior to answering, perhaps it could give even better answers.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#28

Post by Darthy »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

Here's a thought: If you let ChatGPT be idle, thinking to itself, dreaming and planning, this might actually cross the boundary towards sentience - of what we would call somebody who is alive. So there might be some safety and moral concerns.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#29

Post by anigbrowl »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

I've tried this approach, but it's not allowed to create buffers and stream text into it for subsequent analysis/evaluation. What I found worked well instead was asking it to outline 3-4 alternatives as bullet points, and then explore different ways of prioritizing them. You can complexify the conversation by assigning labels and making the subject of instructions. This works well for small specific tasks but of course starts to break down with more abstract or general concepts.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#30

Post by basch »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

I’ve gotten it into a somewhat glitched mode after a bunch of turns where it starts to print an inner monologue before it’s response. It does tend to lead to better answers when it has an intermediate thought.
Post reply on HN