Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Pro Model Card [pdf]

storage.googleapis.com

Post reply on HN
Print view View original

151–160 of 359 posts

Re: Gemini 3 Pro Model Card [pdf]

#151

Post by raducu »

tosh wrote: post #90
mynti wrote: post #75

It is interesting that the Gemini 3 beats every other model on these benchmarks, mostly by a wide margin, but not on SWE Bench. Sonnet is still king here and all three look to be basically on the same level. Kind of wild to see them hit such a wall when it comes to agentic coding

This might also hint at SWE struggling to capture what “being good at coding” means. Evals are hard.

> This might also hint at SWE struggling to capture what “being good at coding” means.

My take would be that coding itself is hard, but I'm a software engineer myself so I'm biased.

Re: Gemini 3 Pro Model Card [pdf]

#152

Post by senordevnyc »

amarcheschi wrote: post #29
senordevnyc wrote: post #24

Earlier quoted context omitted.

It hasn't been released, this is just a leak

On reddit I see it's already available on cursor https://www.reddit.com/r/Bard/comments/1p093fb/gemini_3_in_c...

Interesting, it doesn't show up for me in Cursor yet.

Re: Gemini 3 Pro Model Card [pdf]

#153

Post by zamadatix »

falcor84 wrote: post #108
manmal wrote: post #70

Earlier quoted context omitted.

Looks like it will be on par with the contenders when it comes to coding. I guess improvements will be incremental from here on out.

> I guess improvements will be incremental from here on out. What do you mean? These coding leaderboards were at single digits about a year ago and are now in the seventies. These frontier models are arguably already better at the benchmark that any single human - it's unlikely that any particular human dev is knowledgeable to tackle the full range of diverse tasks even in the smaller SWE-Bench Verified within a reas…

A new benchmark comes out, it's designed so nothing does well at it, the models max it out, and the cycle repeats. This could either describe massive growth of LLM coding abilities or a disconnect between what the new benchmarks are measuring & why new models are scoring well after enough time. In the former assumption there is no limit to the growth of scores... but there is also not very much actual growth (if any at all). In the latter the growth matches, but the reality of using the tools does not seem to say they've actually gotten >10x better at writing code for me in the last year.

Whether an individual human could do well across all tasks in a benchmark is probably not the right question to be asking a benchmark to measure. It's quite easy to construct benchmark tasks a human can't do well in that you don't even need AI to do better.

Re: Gemini 3 Pro Model Card [pdf]

#154

Post by _factor »

felipeerias wrote: post #117
mynti wrote: post #75

It is interesting that the Gemini 3 beats every other model on these benchmarks, mostly by a wide margin, but not on SWE Bench. Sonnet is still king here and all three look to be basically on the same level. Kind of wild to see them hit such a wall when it comes to agentic coding

IMHO coding use cases are much more constrained by tooling than by raw model capabilities at the moment. Perhaps we have finally reached the time of diminishing returns and that will remain the case going forward.

This seems preferable. Wasting tokens on tools when a standardized, reliable interface to those tools should be all that's required.

The magic of LLMs is that they can understand the latent space of a problem and infer a mostly accurate response. Saying you need to subscribe to get the latest tools is just a sales tactic trained into the models to protect profits.

Re: Gemini 3 Pro Model Card [pdf]

#156

Post by senordevnyc »

margorczynski wrote: post #68
senordevnyc wrote: post #22

Earlier quoted context omitted.

1) New SOTA models come out all the time and that hasn't killed the other major AI companies. This will be no different. 2) Google's search revenue last quarter was $56 billion, a 14% increase over Q3 2024.

1) Not long ago Altman and the OpenAI CFO were openly asking for public money. None of these AI companies have actually any kind of working business plan and are just burning investor money. If the investors see there is no winning against Google (or some open Chinese model) the money will dry up. 2) I'm not suggesting this will happen overnight but especially younger people gravitate towards LLM for information sear…

No, you suggested that LLMs are clearly eating google's lunch already, and there's just no evidence of that. Quite the opposite.

Re: Gemini 3 Pro Model Card [pdf]

#157

Post by netdevphoenix »

llm_nerd wrote: post #102
margorczynski wrote: post #11

If these numbers are true then OpenAI is probably done, Anthropic too. Still, it's hard to see an effective monetization method for this tech and it clearly is eating Google's main pie which is search.

They're constantly matching and exceeding each other. It's a hypercompetitive space and I would fully expect one of the others to top various benchmarks shortly after. On pretty much every leading release someone does this "everyone else is done! Shut er down" thing and it's growing pretty weird. Having said that, OpenAI's ridiculous hype cycle has been living on borrowed time. OpenAI has zero moat, and are just one…

This. If I had to put my money on a survivor, it would be Google because it is an established company with existing revenue modules unrelated to AI. Anthropic and OpenAI won't stand alone without external funding

Re: Gemini 3 Pro Model Card [pdf]

#158

Post by mikert89 »

api wrote: post #129
catigula wrote: post #63

I know this is a little controversial but the lack of performance on SWE-bench is hugely disappointing I think economically. These models don’t have any viable path to profitability if they can’t take engineering jobs.

Really? If they can make an engineer more productive, that's worth a lot. Naive napkin math: 1.5X productivity on one $200k/year engineer is worth $100k/year.

People generally dont understand what these models are doing to engineering salaries. The skill level required to produce working software is going way down

Re: Gemini 3 Pro Model Card [pdf]

#159

Post by mimentum »

jmkni wrote: post #69
denysvitali wrote: post #59

Title of the document is "[Gemini 3 Pro] External Model Card - November 18, 2025 - v2", in case you needed further confirmation that the model will be released today. Also interesting to know that Google Antigravity (antigravity.google / https://github.com/Google-Antigravity ?) leaked. I remember seeing this subdomain recently. Probably Gemini 3 related as well. Org was created on 2025-11-04T19:28:13Z ( https://api.g…

what is Google Antigravity?

According to Gemini itself:

"Google Antigravity" refers to a new AI software platform announced by Google designed to help developers write and manage code.

The term itself is a bit of a placeholder or project name, combining the brand "Google" with the concept of "antigravity"—implying a release from the limitations of traditional coding.

In simple terms, Google Antigravity is a sophisticated tool for programmers that uses powerful AI systems (called "agents") to handle complex coding tasks automatically. It takes the typical software workbench (an IDE) and evolves it into an "agent-first" system.

Agentic Platform: It's a central hub where many specialized AI helpers (agents) live and work together. The goal is to let you focus on what to build, not how to build it.

Task-Oriented: The platform is designed to be given a high-level goal (a "task") rather than needing line-by-line instructions.

Autonomous Operation: The AI agents can work across all your tools—your code editor, the command line, and your web browser—without needing you to constantly supervise or switch between them.

Re: Gemini 3 Pro Model Card [pdf]

#160

Post by danielcampos93 »

scrlk wrote: post #23

Benchmarks from page 4 of the model card: | Benchmark | 3 Pro | 2.5 Pro | Sonnet 4.5 | GPT-5.1 | |-----------------------|-----------|---------|------------|-----------| | Humanity's Last Exam | 37.5% | 21.6% | 13.7% | 26.5% | | ARC-AGI-2 | 31.1% | 4.9% | 13.6% | 17.6% | | GPQA Diamond | 91.9% | 86.4% | 83.4% | 88.1% | | AIME 2025 | | | | | | (no tools) | 95.0% | 88.0% | 87.0% | 94.0% | | (code execution) | 100% | -…

I would love to know what the increased token count is across these models for the benchmarks. I find the models continue to get better but as they do their token usage also does. Aka is model doing better or reasoning for longer?
Post reply on HN