Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Deep Think

blog.google

Post reply on HN
Print view View original

1–10 of 722 posts

Re: Gemini 3 Deep Think

#2

Post by Metacelsus »

According to benchmarks in the announcement, healthily ahead of Claude 4.6. I guess they didn't test ChatGPT 5.3 though.

Google has definitely been pulling ahead in AI over the last few months. I've been using Gemini and finding it's better than the other models (especially for biology where it doesn't refuse to answer harmless questions).

Re: Gemini 3 Deep Think

#3

Post by sigmar »

Here is the methodologies for all the benchmarks: https://storage.googleapis.com/deepmind-media/gemini/gemini_...

The arc-agi-2 score (84.6%) is from the semi-private eval set. If gemini-3-deepthink gets above 85% on the private eval set, it will be considered "solved"

>Submit a solution which scores 85% on the ARC-AGI-2 private evaluation set and win $700K. https://arcprize.org/guide#overview

Re: Gemini 3 Deep Think

#5

Post by simianwords »

Metacelsus wrote: post #2

According to benchmarks in the announcement, healthily ahead of Claude 4.6. I guess they didn't test ChatGPT 5.3 though. Google has definitely been pulling ahead in AI over the last few months. I've been using Gemini and finding it's better than the other models (especially for biology where it doesn't refuse to answer harmless questions).

The comparison should be with GPT 5.2 pro which has been used successfully to solve open math problems.

Re: Gemini 3 Deep Think

#6

Post by simianwords »

OT but my intuition says that there’s a spectrum

- non thinking models

- thinking models

- best of N models like deep think an gpt pro

Each one is of a certain computational complexity. Simplifying a bit, I think they map to - linear, quadratic and n^3 respectively.

I think there are certain class of problems that can’t be solved without thinking because it necessarily involves writing in a scratchpad. And same for best of N which involves exploring.

Two open questions

1) what’s the higher level here, is there a 4th option?

2) can a sufficiently large non thinking model perform the same as a smaller thinking?

Re: Gemini 3 Deep Think

#10

Post by gs17 »

sigmar wrote: post #3

Here is the methodologies for all the benchmarks: https://storage.googleapis.com/deepmind-media/gemini/gemini_... The arc-agi-2 score (84.6%) is from the semi-private eval set. If gemini-3-deepthink gets above 85% on the private eval set, it will be considered "solved" >Submit a solution which scores 85% on the ARC-AGI-2 private evaluation set and win $700K. https://arcprize.org/guide#overview

Interestingly, the title of that PDF calls it "Gemini 3.1 Pro". Guess that's dropping soon.
Post reply on HN