ChatGPT has trouble giving an answer before explaining its reasoning

yes, hopefully I'll write it up soon. TL;DR: I used this top of GPT-3.5 to generate a magazine page of personalized recommendations: 3x of a title, paragraph, AI art, and a font name, and rationale. For images, I use SD 2.1 via stability.ai. Be sure to add 5400 dpi digital art at the front of your prompt :)

I'm a bit embarrassed to, "real" research finetunes internal models to play a particular role, rather than orchestrating several "conversations" and hoping your prompt will get you the right output format 100% of the time, etc.

Here's a woefully lacking diagram of this user/interpreter/LLM flow for a cohesive longform story generator. [1]

The coolest part of this design pattern you've ID'd is you can always add one more character/conversation that the interpreter orchestrates

ex. A DB character whose role is taking a new page as input, then outputting the new DB, where the DB is all important facts to sustain over a story. That let me scale to 16+ "pages"

[1] https://twitter.com/jpohhhh/status/1632082749317054468?s=20

I just had this interaction with ChatGPT.

Me: Reverse the digits of 12+39

ChatGPT: The sum of 12 and 39 is 51. If you reverse the digits, you get 15.

Me: Reverse the digits of 12 + 84. Only respond with the reversed digits, no explanation

ChatGPT: The reversed digits of 12 + 84 are 96.

Which makes me think that longer explanations give it more of a chance to think because it gets more passes through the model. Weird!

Bing Chat according to the leaks[1] uses an inner monologue.

I remember other people getting similar results, which suggests it's not an hallucination

[1] https://www.reddit.com/r/bing/comments/11ironc/bing_reveals_...

On the "pound of potatoes vs pound of feathers" question - ChatGPT's explanation is so convincing that it really understands the concepts. But it can be juked very easily.

I asked it what was heavier, 1,000 tons of helium or a cubic meter of osmium, and it told me that the helium weighs 1,000,000 kg, the osmium 22,600 kg, and therefore the osmium weighs more because it's way denser.

It's hard for me to believe that these LLMs are going to learn basic logic if only we throw more training data at them.

Transcript:

Me: What is heavier, 1000 tons of helium or a cubic meter of osmium?

ChatGPT: A cubic meter of osmium is much heavier than 1000 tons of helium.

One ton is equal to 1000 kilograms, so 1000 tons is equal to 1,000,000 kilograms. The density of helium is about 0.1785 kilograms per cubic meter. Therefore, 1000 tons of helium would have a volume of:

1,000,000 kg / 0.1785 kg/m³ = 5,604,069.77 m³

On the other hand, osmium is one of the densest elements known, with a density of around 22,600 kg/m³. Therefore, a cubic meter of osmium weighs:

1 m³ x 22,600 kg/m³ = 22,600 kg

So, a cubic meter of osmium weighs much more than 1000 tons of helium.

Update: I am continuing the conversation and I am now being explicitly told that, while 1,000,000 kg is much heavier than 22,600 kg, it doesn't change the fact that the osmium is heavier than the helium because the osmium is denser.

Update2: I then reminded it about the potatoes and feathers and how density was irrelevant in that context, and shouldn't it therefore be irrelevant in the case of the helium and the osmium? And instead of correcting its response on the helium and osmium, it's now telling me the feathers and potatoes weigh different.

Update3: it is now telling me that densities don't matter when comparing masses but do matter when comparing weights. I must say, it has a certain panache in resolving internal inconsistencies in its past responses.

Update4: after being corrected half a dozen times with contradictory information, I asked it to state its confidence in its latest story. It said "I can state with a high degree of confidence that my last answer was accurate". The shamelessness!

It's never going to be great at math problems, it is a language model.