This is called "hallucination" and I find it to be the biggest flaw with GPT-3 interfaces like ChatGPT. Basically, the system will start convincingly BSing at any point and as a user, you're never sure if any particular detail it outputs is actually correct. The problem is, these large language models are inherently kind of a black box, so how do you fix this kind of behavior? I started looking and it turns out that…
Is it possible to give something like ChatGPT an understanding of what it's talking about? I feel like there's something fundamental missing for the model to be something more than just a neat way to remix language.