Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Pro Model Card [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

251–260 of 359 posts

Re: Gemini 3 Pro Model Card [pdf]

#251

Post by stego-tech »

rvnx wrote: post #249
spoaceman7777 wrote: post #183

Earlier quoted context omitted.

Wow. They must have had some major breakthrough. Those scores are truly insane. O_O Models have begun to fairly thoroughly saturate "knowledge" and such, but there are still considerable bumps there But the _big news_, and the demonstration of their achievement here, are the incredible scores they've racked up here for what's necessary for agentic AI to become widely deployable. t2-bench. Visual comprehension. Comput…

The problem is that we know in advance what is the benchmark, so Humanity's Last Exam for example, it's way easier to optimize your model when you have seen the questions before.

This. A lot of boosters point to benchmarks as justification of their claims, but any gamer who spent time in the benchmark trenches will know full well that vendors game known tests for better scores, and that said scores aren’t necessarily indicative of superior performance. There’s not a doubt in my mind that AI companies are doing the same.

Re: Gemini 3 Pro Model Card [pdf]

#252

Post by Feuilles_Mortes »

rvnx wrote: post #249
spoaceman7777 wrote: post #183

Earlier quoted context omitted.

Wow. They must have had some major breakthrough. Those scores are truly insane. O_O Models have begun to fairly thoroughly saturate "knowledge" and such, but there are still considerable bumps there But the _big news_, and the demonstration of their achievement here, are the incredible scores they've racked up here for what's necessary for agentic AI to become widely deployable. t2-bench. Visual comprehension. Comput…

The problem is that we know in advance what is the benchmark, so Humanity's Last Exam for example, it's way easier to optimize your model when you have seen the questions before.

shouldn't we expect that all of the companies are doing this optimization, though? so, back to level playing field.

Re: Gemini 3 Pro Model Card [pdf]

#253

Post by denysvitali »

jmkni wrote: post #69
denysvitali wrote: post #59

Title of the document is "[Gemini 3 Pro] External Model Card - November 18, 2025 - v2", in case you needed further confirmation that the model will be released today. Also interesting to know that Google Antigravity (antigravity.google / https://github.com/Google-Antigravity ?) leaked. I remember seeing this subdomain recently. Probably Gemini 3 related as well. Org was created on 2025-11-04T19:28:13Z ( https://api.g…

what is Google Antigravity?

> Google Antigravity is an agentic development platform, evolving the IDE into the agent-first era. Antigravity enables developers to operate at a higher, task-oriented level by managing agents across workspaces, while retaining a familiar AI IDE experience at its core. Agents operate across the editor, terminal, and browser, enabling them to autonomously plan and execute complex, end-to-end tasks elevating all aspects of software development.

Now the page is somewhat live on that URL

Re: Gemini 3 Pro Model Card [pdf]

#254

Post by bretpiatt »

Page 5, "The knowledge cutoff date for Gemini 3 Pro was January 2025."

Still taking nearly a year to train and run post training safety and stability tuning.

With 10x the infrastructure they could iterate much faster, I don't see AI infrastructure as a bubble, it is still a bottleneck on pace of innovation at today's active deployment level.

Re: Gemini 3 Pro Model Card [pdf]

#255

Post by babl-yc »

ethmarks wrote: post #187

> TPUs are specifically designed to handle the massive computations involved in training LLMs and can speed up training considerably compared to CPUs. That seems like a low bar. Who's training frontier LLMs on CPUs? Surely they meant to compare TPUs to GPUs. If "this is faster than a CPU for massively parallel AI training" is the best you can say about it, that's not very impressive.

I don't know if you can generally say that "LLM training is faster on TPUs vs GPUs". There is variance among LLM architectures, TPU cluster sizes, GPU cluster sizes...

They are both designed to do massively parallel operations. TPUs are just a bit more specific to matrix multiply+adds while GPUs are more generic.

Re: Gemini 3 Pro Model Card [pdf]

#257

Post by HardCodedBias »

postalcoder wrote: post #79
Alifatisk wrote: post #71

Earlier quoted context omitted.

These numbers are impressive, at least to say. It looks like Google has produced a beast that will raise the bar even higher. What's even more impressive is how Google came into this game late and went from producing a few flops to being the leader at this (actually, they already achieved the title with 2.5 Pro). What makes me even more curious is the following > Model dependencies: This model is not a modification o…

Google was never really late. Where people perceived Google to have dropped the ball was in its productization of AI. The Google's Bard branding stumble was so (hilariously) bad that it threw a lot of people off the scent. My hunch is that, aside from "safety" reasons, the Google Books lawsuit left some copyright wounds that Google did not want to reopen.

Bard was horrible compared to the competition of the time.

Gemini 1.0 was strictly worse than GPT-3.5 and was unusable due to "safety" features.

Google followed that up with 1.5 which was still worse than GPT-3.5 and unbelievably far behind GPT-4. At this same time Google had their "black nazi" scandals.

With Gemini 2.0 finally had a model that was at least useful for OCR and with their fash series a model that, while not up to par in capabilities, was sufficiently inexpensive that it found uses.

Only with Gemini-2.5 did Google catch up with SoTA. It was within "spitting distance" of the leading models.

Google did indeed drop the ball, very, very badly.

I suspect that Sergey coming back helped immensely, somehow. I suspect that he was able to tame some of the more dysfunctional elements of Google, at least for a time.

Re: Gemini 3 Pro Model Card [pdf]

#258

Post by Workaccount2 »

dgacmu wrote: post #120
postalcoder wrote: post #79

Earlier quoted context omitted.

Google was never really late. Where people perceived Google to have dropped the ball was in its productization of AI. The Google's Bard branding stumble was so (hilariously) bad that it threw a lot of people off the scent. My hunch is that, aside from "safety" reasons, the Google Books lawsuit left some copyright wounds that Google did not want to reopen.

The memory of Microsoft's Tay fiasco was strong around the time the brain team started playing with chatbots.

Google was catastrophically traumatized throughout the org when they had that photos AI mislabel black people as gorillas. They turned the safety and caution knobs up to 12 after that for years, really until OpenAI came along and ate their lunch.

Re: Gemini 3 Pro Model Card [pdf]

#259

Post by camdenreslink »

bretpiatt wrote: post #254

Page 5, "The knowledge cutoff date for Gemini 3 Pro was January 2025." Still taking nearly a year to train and run post training safety and stability tuning. With 10x the infrastructure they could iterate much faster, I don't see AI infrastructure as a bubble, it is still a bottleneck on pace of innovation at today's active deployment level.

But if they spend 10x on infrastructure, and capabilities only improve 10%, then that still can be a bubble even if infrastructure is a bottleneck.

Re: Gemini 3 Pro Model Card [pdf]

#260

Post by HardCodedBias »

scrlk wrote: post #23

Benchmarks from page 4 of the model card: | Benchmark | 3 Pro | 2.5 Pro | Sonnet 4.5 | GPT-5.1 | |-----------------------|-----------|---------|------------|-----------| | Humanity's Last Exam | 37.5% | 21.6% | 13.7% | 26.5% | | ARC-AGI-2 | 31.1% | 4.9% | 13.6% | 17.6% | | GPQA Diamond | 91.9% | 86.4% | 83.4% | 88.1% | | AIME 2025 | | | | | | (no tools) | 95.0% | 88.0% | 87.0% | 94.0% | | (code execution) | 100% | -…

Big if true.

I'll wait for the official blog with benchmark results.

I suspect that our ability to benchmark models is waning. Much more investment required in this area, but what is the play out?

Post reply on HN