> This is some careful writing. Most important are the weasel words,
So what do you want, a full refund? OpenAI is probably pretty busy, so I'll handle that for them and give you your nothing back. Glad to be of service. :P
> It would be really useful if we could just teach it to do the second one, right? But we can't, because ChatGPT never draws a distinction between grammar and subjects.
The structure of the world, as expressed in its training data, is part of it's 'grammar'. It absolute can and does favor giving truthful answers even though a lie is equally grammatical English.
One only need spend a minute looking at HN though to see that "I don't know" is really not part of the language used in most places online, so far they haven't come up with a way of training the model on all the places where someone saw something and decided to say nothing. :)
> In a word: training.
We all also learn about the world through training. The LLMs aren't at all like human minds-- they're fixed depth circuits and can only engage in longer thoughts by 'thinking out loud', as an example. But to say it can't distinguish truth from lies because both are valid language sounds like an error resulting from hearing the title "language model". LLMs model the world through language.