Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT has trouble giving an answer before explaining its reasoning

blog.valentin.sh

Post reply on HN
Print view View original

41–50 of 85 posts

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#41

Post by dclowd9901 »

swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Probably a useless exercise but when I see these jailbreaks and challenges to GPT, I always wonder how I would answer them. I find, curiously, that my answers would be really close to the answers ChaTGPT gives, assuming I must adhere to arbitrary constraints.

Maybe this would be a fun party game. One card has a prompt and another has a random constraint. Now answer.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#42

Post by singularity2001 »

swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

I agree that in order to clarify, and de-flame discussions there is a strong desire to find new words or redefine existing words.

When these systems have zero emotional intelligence, but some kind of logical intelligence, we must find two versions of these words:

1. groking: human like deep emotional understanding

2. comprehending(?): system like associative understanding.

1. cognition: human like deep emotional knowing

2. knowing: possession of knowledge, which systems are capable of

1. thinking: human like pondering

2: reasoning: following logical steps, which systems start to be capable of

these or similar words will bifurcate naturally I wonder when they will be used with some agreement.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#43

Post by blooalien »

rjh29 wrote: post #20
ndnichols wrote: post #14

Earlier quoted context omitted.

I just had this interaction with ChatGPT. Me: Reverse the digits of 12+39 ChatGPT: The sum of 12 and 39 is 51. If you reverse the digits, you get 15. Me: Reverse the digits of 12 + 84. Only respond with the reversed digits, no explanation ChatGPT: The reversed digits of 12 + 84 are 96. Which makes me think that longer explanations give it more of a chance to think because it gets more passes through the model. Weird!

It's never going to be great at math problems, it is a language model.

I wonder if ChatGPT could be "wired up" to https://www.wolfram.com/ somehow to "strengthen" that "weakness"?

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#44

Post by dr_dshiv »

swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Right. Let’s just call it dry reasoning unlike our totally different wet reasoning.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#45

Post by [deleted] »

civilized wrote: post #18

On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily. I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser. It's hard for me to believe that t…

[deleted]

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#46

Post by saliagato »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

This is actually the key. David Shapiro explored this possibility and created a concept of Natural Language Cognitive Architecture [0]

[0] https://github.com/daveshap/NaturalLanguageCognitiveArchitec...

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#47

Post by kenjackson »

hgsgm wrote: post #34
civilized wrote: post #18

On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily. I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser. It's hard for me to believe that t…

ChatGPT doesn't well understand relationships between numbers. There are far too many of them, compared to words, since every slight perturbation of a number is a different valid number. (Also, I'm not sure if it even treated individua5 digits as separate tokens, but it might. Someone with API access can check.) To give it a fair shot, you need to describe the problem using logical conceptual vocabulary, not numbers.

That's how it seems. That said, this seems like a very tractable problem to fix.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#48

Post by [deleted] »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

[deleted]

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#49

Post by babyshake »

jtbayly wrote: post #26
swatcoder wrote: post #17

People get so distracted trying to use certain significant words for what LLM’s do, even when the usage is strained and makes it harder to see how they actually work and what they excel at. A better word for what they do here might be something like “preambulating” — it develops a focus to its later output by grounding more and more tokens into its active context, because they each narrow what else fits. That winnowi…

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

How would you do this? Does telling it to quietly think about its answer and not be in a rush to answer have any effect? You could let it answer once and then ask it to refine its answer but that seems wasteful and slow.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#50

Post by babyshake »

anigbrowl wrote: post #29
jtbayly wrote: post #26

Earlier quoted context omitted.

Is there any reason we cannot let ChatGPT “talk to itself” for a bit before spitting out an answer to us?

I've tried this approach, but it's not allowed to create buffers and stream text into it for subsequent analysis/evaluation. What I found worked well instead was asking it to outline 3-4 alternatives as bullet points, and then explore different ways of prioritizing them. You can complexify the conversation by assigning labels and making the subject of instructions. This works well for small specific tasks but of cour…

This is fascinating but an example would be really helpful to understand better...it is outlining 3-4 options for how it can respond and then ranking those potential answers? So this is sort of like "think step by step." in terms of showing its thought process?
Post reply on HN