Earlier quoted context omitted.
Of course there is strong correlation. That is literally what it was designed to do. The problem is that it will simultaneously say that "cow eggs are bigger than chicken eggs", with the same confidence (and in a way that correlates well with human evaluators). https://www.reddit.com/r/Funnymemes/comments/10ohd2n/chatgpt... So when you get an evaluation you are playing the russian roulette - you may get a decent resu…
I just asked and it told me cows are mammals and do not lay eggs. That Reddit post is not even GPT-4 and is 5 months old, which may as well be the 19th century on AI tech timescales.
The point is that the tool fails, and it is known to fail, so much that we even have a name for the times when it fails - hallucinations. I have been calling them cow eggs because that's a nice mental image and I didn't want to have to remember for the proper English term. I will continue calling them cow eggs.