Earlier quoted context omitted.
>As GPT-4 had been shown to outperform humans on the coding competition tasks On the same frontpage: https://news.ycombinator.com/item?id=35297067 Food for thought. Also by their own paper it has only a codeforces rank of 392 which is below 5th percentile.
Also a cute prompt I saw on the same twitter thread ``` This is a variant of the monty hall problem: Suppose you're on a game show, and you're given the choice of three doors. IMPORTANT: All the doors are transparent. Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who clearly knows what's behind the doors, opens another door, say No. 3, which has a goat. He then says to…
The problem is that as soon as people started tricking ChatGPT 3 into problems like that, the correct answers are now being used to train the next versions and are going to be part of the dataset.
So GPT-4 or GPT-5 may get the answer right, but that still wouldn't mean anything.