The pelican riding a bicycle is excellent . I think it's the best I've seen. https://simonwillison.net/2026/Feb/12/gemini-3-deep-think/
How likely this problem is already on the training set by now?
Gemini 3 Deep Think
41–50 of 722 posts
Re: Gemini 3 Deep Think
#42The pelican riding a bicycle is excellent . I think it's the best I've seen. https://simonwillison.net/2026/Feb/12/gemini-3-deep-think/
The reflection of the sun in the water is completely wrong. LLMs are still useless. (/s)
Re: Gemini 3 Deep Think
#43Re: Gemini 3 Deep Think
#44Why a Twitter post and not the official Google blog post… https://blog.google/innovation-and-ai/models-and-research/ge...
Just normal randomness I suppose. I've put that URL at the top now, and included the submitted URL in the top text.
Re: Gemini 3 Deep Think
#45Here is the methodologies for all the benchmarks: https://storage.googleapis.com/deepmind-media/gemini/gemini_... The arc-agi-2 score (84.6%) is from the semi-private eval set. If gemini-3-deepthink gets above 85% on the private eval set, it will be considered "solved" >Submit a solution which scores 85% on the ARC-AGI-2 private evaluation set and win $700K. https://arcprize.org/guide#overview
> If gemini-3-deepthink gets above 85% on the private eval set, it will be considered "solved"
They never will do on private set, because it would mean its being leaked to google.
Re: Gemini 3 Deep Think
#46The pelican riding a bicycle is excellent . I think it's the best I've seen. https://simonwillison.net/2026/Feb/12/gemini-3-deep-think/
How likely this problem is already on the training set by now?
I've heard it posited that the reason the frontier companies are frontier is because they have custom data and evals. This is what I would do too
Re: Gemini 3 Deep Think
#47The pelican riding a bicycle is excellent . I think it's the best I've seen. https://simonwillison.net/2026/Feb/12/gemini-3-deep-think/
How likely this problem is already on the training set by now?
For every combination of animal and vehicle? Very unlikely.
The beauty of this benchmark is that it takes all of two seconds to come up with your own unique one. A seahorse on a unicycle. A platypus flying a glider. A man’o’war piloting a Portuguese man of war. Whatever you want.
Re: Gemini 3 Deep Think
#48Re: Gemini 3 Deep Think
#49Re: Gemini 3 Deep Think
#50Earlier quoted context omitted.
How likely this problem is already on the training set by now?
For every combination of animal and vehicle? Very unlikely. The beauty of this benchmark is that it takes all of two seconds to come up with your own unique one. A seahorse on a unicycle. A platypus flying a glider. A man’o’war piloting a Portuguese man of war. Whatever you want.
No, not every combination. The question is about the specific combination of a pelican on a bicycle. It might be easy to come up with another test, but we're looking at the results from a particular one here.