It's so weird to me when apparently tech-savvy people say ChatGPT "lies", or call it a bullshit generator and so on. I mean - when you ask StableDiffusion to draw a dog astronaut, everyone gets that the image it returns is made-up, right? Nobody expects the AI to return only "true" images of existing things - it was trained on fictional images as well as photographs, and people understand that it can imagine new thin…
It’s generally accepted that artwork can delve into the fantastical and absurd. That’s part of the point of it being art.
ChatGPT isn’t for the most part trying to be performance artwork - it is striving to be ‘right’ and to give the correct answer to your question.