Can GPT-5 Beat My Favorite Daily Puzzle Game?

#1

Post by nsypteras » Fri, Nov 14, 2025, 5:08 PM UTC

Can GPT-5 Beat My Favorite Daily Puzzle Game?
nicksypteras.com

Re: Can GPT-5 Beat My Favorite Daily Puzzle Game?

#2

Post by pyankoff » Fri, Nov 14, 2025, 5:17 PM UTC

Very cool! The massive outperformance of GPT-5 looks like there is something different in their training data indeed. Considering their previous work on games, wouldn't be surprising if they generated some synthetic game data.

Re: Can GPT-5 Beat My Favorite Daily Puzzle Game?

#3

Post by srekhi » Fri, Nov 14, 2025, 5:29 PM UTC

interesting - and thx for making reproducible

Re: Can GPT-5 Beat My Favorite Daily Puzzle Game?

#4

Post by nsypteras » Fri, Nov 14, 2025, 5:35 PM UTC

pyankoff wrote: post #2

Very cool! The massive outperformance of GPT-5 looks like there is something different in their training data indeed. Considering their previous work on games, wouldn't be surprising if they generated some synthetic game data.

Ya interesting thought - would be fascinating if generating games w/solutions is part of the training data pipeline. There's been previous work done on on testing LLMs on logic puzzles[1][2][3] so they could possibly be building off those ideas to improve performance.

[1] https://huggingface.co/papers/2504.00043 [2] https://huggingface.co/blog/yuchenlin/zebra-logic [3] https://arxiv.org/pdf/2403.12094

Re: Can GPT-5 Beat My Favorite Daily Puzzle Game?

#5

Post by sonnynomnom » Fri, Nov 14, 2025, 11:54 PM UTC

grok 4's results lolol