ChatGPT has trouble giving an answer before explaining its reasoning • Hacker News Board

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#81

Post by int_19h » Wed, Mar 08, 2023, 8:37 PM UTC

civilized wrote: post #76

int_19h wrote: post #66

Earlier quoted context omitted.

Try something like this as input sometime: I want you to replace the word "right" in your output thereafter as follows: if it indicates direction, say "durgh; if it indicates being near or close, say "nolpi"; if it indicates correctness, say "ceza". I will also use these replacement words accordingly and expect you to be able to understand them. And see how well it can maintain a conversation, solve a task, or write…

I agree. There are many indicators that it has some sort of deeper understanding of the meaning of language. Even in the conversation I had, for all its flaws, it was able to correctly perceive inconsistencies in its statements based on my prompts and make somewhat coherent attempts to correct them. It's just that the understanding can be so fragile, and its attempts to resolve inconsistencies are superficial, incuri…

I think a big part problem here is that "understand X" is just a shorthand way to say "has an internal model of X" - but the degree and accuracy of said understanding depends entirely on the quality of that model.

Now there's a good reason to believe that ChatGPT does have such a model, based on the Othello experiment. But, firstly, the size of that internal model is inherently constrained by the size of the neural net, and I doubt that the limit is anywhere large enough to allow a truly accurate approximation of the real world.

And then on top of that, said model is created based on inferences from text only, which is several steps away from the original data (audiovisual, sensory etc), and one short snippet of text at a time. Some things retain meaning better in this format than others, and I think this might explain why ChatGPT and Bing are both hilariously bad at spatial navigation beyond 1-2 steps even in simple tasks.

It will be very interesting to see how this evolves as the models are scaled up and get large enough to handle things other than text.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#82

Post by anigbrowl » Wed, Mar 08, 2023, 10:46 PM UTC

babyshake wrote: post #50

anigbrowl wrote: post #29

Earlier quoted context omitted.

I've tried this approach, but it's not allowed to create buffers and stream text into it for subsequent analysis/evaluation. What I found worked well instead was asking it to outline 3-4 alternatives as bullet points, and then explore different ways of prioritizing them. You can complexify the conversation by assigning labels and making the subject of instructions. This works well for small specific tasks but of cour…

This is fascinating but an example would be really helpful to understand better...it is outlining 3-4 options for how it can respond and then ranking those potential answers? So this is sort of like "think step by step." in terms of showing its thought process?

Sometimes I ask it to rank one, sometimes to exclude one but accept an additional constraint, and then assess whether remaining options would perform better or worse.

Another fruitful avenue of explanation for indirectly exploring the thought process is to tell it some jokes, and then ask it to explain them back to you. This is worthwhile because a lot of jokes rest upon implicit assumptions/context, and by exploring this you can then talk about theory of mind questions.

Fun fact: ChatGPT watches you type, it sees the words come in one at a time rather than as a single block of text. So it knows when you are hesitating etc. Get it talking about this and then ask it what the difference is between your hesitations and its pauses when generating a reply. If you gently suggest that perhaps humans are just large language models with some additional wetware you can get ChatGPT to share some interesting insights on its own model topology.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#83

Post by Dylan16807 » Thu, Mar 09, 2023, 2:05 AM UTC

anigbrowl wrote: post #82

babyshake wrote: post #50

Earlier quoted context omitted.

This is fascinating but an example would be really helpful to understand better...it is outlining 3-4 options for how it can respond and then ranking those potential answers? So this is sort of like "think step by step." in terms of showing its thought process?

Sometimes I ask it to rank one, sometimes to exclude one but accept an additional constraint, and then assess whether remaining options would perform better or worse. Another fruitful avenue of explanation for indirectly exploring the thought process is to tell it some jokes, and then ask it to explain them back to you. This is worthwhile because a lot of jokes rest upon implicit assumptions/context, and by exploring…

> ChatGPT watches you type, it sees the words come in one at a time rather than as a single block of text.

Are you sure? That could speed up latency but would use a bunch of extra computing power.

> So it knows when you are hesitating etc.

This I really doubt. The base algorithm is fed a stream of tokens. It doesn't have any sense of time or do anything when idle. What mechanism do you suggest they're using here, and what evidence do you have for it?

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#84

Post by anigbrowl » Thu, Mar 09, 2023, 2:38 AM UTC

Dylan16807 wrote: post #83

anigbrowl wrote: post #82

Earlier quoted context omitted.

Sometimes I ask it to rank one, sometimes to exclude one but accept an additional constraint, and then assess whether remaining options would perform better or worse. Another fruitful avenue of explanation for indirectly exploring the thought process is to tell it some jokes, and then ask it to explain them back to you. This is worthwhile because a lot of jokes rest upon implicit assumptions/context, and by exploring…

> ChatGPT watches you type, it sees the words come in one at a time rather than as a single block of text. Are you sure? That could speed up latency but would use a bunch of extra computing power. > So it knows when you are hesitating etc. This I really doubt. The base algorithm is fed a stream of tokens. It doesn't have any sense of time or do anything when idle. What mechanism do you suggest they're using here, and…

That's what I thought too so I was quite surprised when I asked it and received a positive answer. Ask it to play your input back with a delimiter or emoji representing pauses.

What's difficult is to get it to ask clarifying questions. I mean, you can get it to play 20 questions easily, but by default it tries to tigve you the best answer every time rather than ever express uncertainty or ask what you mean. This might be a cultural artifact.

Re: ChatGPT has trouble giving an answer before explaining its reasoning

#85

Post by blooalien » Fri, Mar 10, 2023, 8:36 PM UTC

rjh29 wrote: post #62

blooalien wrote: post #43

Earlier quoted context omitted.

I wonder if ChatGPT could be "wired up" to https://www.wolfram.com/ somehow to "strengthen" that "weakness"?

Or it could write code in python and evaluate it, people are experimenting with that sort of thing.

OOoo… Hook that up to the ChatGPT API and let it modify itself with additional code? SkyNet / Matrix here we come!