Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.1 Pro

blog.google

Post reply on HN
Print view View original

611–620 of 951 posts

Re: Gemini 3.1 Pro

#611

Post by andai »

bluegatty wrote: post #589
spankalee wrote: post #199

I hope this works better than 3.0 Pro I'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development. It's stunningly good at reasoning, design, and generating the raw code, but it just falls over a lot when actually trying to get things done, especially compared to Claude Opus. Within VS Code Copi…

Yes, this is very true and it speaks strongly to this wayward notion of 'models' - it depends so much on the tuning, the harness, the tools. I think it speaks to the broader notion of AGI as well. Claude is definitively trained on the process of coding not just the code, that much is clear. Codex has the same limitation but not quite as bad. This may be a result of Anthropic using 'user cues' with respect to what are…

Tell me more about Codex. I'm trying to understand it better.

I have a pretty crude mental model for this stuff but Opus feels more like a guy to me, while Codex feels like a machine.

I think that's partly the personality and tone, but I think it goes deeper than that.

(Or maybe the language and tone shapes the behavior, because of how LLMs work? It sounds ridiculous but I told Claude to believe in itself and suddenly it was able to solve problems it wouldn't even attempt before...)

Re: Gemini 3.1 Pro

#612

Post by TZubiri »

mikestorrent wrote: post #568
small_model wrote: post #478

Earlier quoted context omitted.

I was being facetious, I mean one day models might skip the middle man of code and compilation and take your specs and produce an ultra efficent binary.

Musk was saying that recently but I don't see it being efficient or worthwhile to do this. I could be proven brutally wrong, but code is language; executables aren't. There's also no real reason to bother with this when we have quick-compiling languages. More realistically, I could see particular languages and frameworks proving out to be more well-designed and apt for AI code creation; for instance, I was always too…

You can map tokens to CPU instructions and train a model on that, that's what they do for input images I think.

I think the main limitation on the current models is not that cpu instructions aren't cpu instructions (even though they can be with .asm), it's that they are causal, the cpu would need to generate a binary entirely from start to finish sequentially.

If we learned something over the last 50 years of programming is that that's hard and that's why we invented programming languages? Why would it be simpler to just generate the machine code, sure maybe an LLM to application can exist, but my money is in that there will be a whole toolchain in the middle, and it will probably be the same old toolchain that we are using currently, an OS, probably Linux.

Isn't it more common that stuff builds on the existing infra instead of a super duper revolution that doesn't use the previous tech stack? It's much easier to add onto rather than start from scratch.

Re: Gemini 3.1 Pro

#613

Post by drivebyhooting »

HardCodedBias wrote: post #487
alphabetting wrote: post #393

Earlier quoted context omitted.

the agentic benchmarks for 3.1 indicate Gemini has caught up. the gains are big from 3.0 to 3.1. For example the APEX-Agents benchmark for long time horizon investment banking, consulting and legal work: 1. Gemini 3.1 Pro - 33.2% 2. Opus 4.6 - 29.8% 3. GPT 5.2 Codex - 27.6% 4. Gemini Flash 3.0 - 24.0% 5. GPT 5.2 - 23.0% 6. Gemini 3.0 Pro - 18.0%

LOL come on man. Let's give it a couple of days since no one believes anything from benchmarks, especially from the Gemini team (or Meta). If we see on HN that people are willing switching their coding environment, we'll know "hot damn they cooked" otherwise this is another wiff by Google.

You can’t put Gemini and Meta in the same sentence. Llama 4 was DOA, and Meta has given up on frontier models. Internally they’re using Claude.

Re: Gemini 3.1 Pro

#614

Post by metadat »

sdeiley wrote: post #548

People underrate Google's cost effectiveness so much. Half price of Opus. HALF. Think about ANY other product and what you'd expect from the competition thats half the price. Yet people here act like Gemini is dead weight ____ Update: 3.1 was 40% of the cost to run AA index vs Opus Thinking AND SONNET, beat Opus, and still 30% faster for output speed. https://artificialanalysis.ai/?speed=intelligence-vs-speed&m...

Attention is the new scarce resource. Saving even 50% is nothing if it wastes more of my time.

Re: Gemini 3.1 Pro

#615

Post by sdeiley »

fastball wrote: post #593
sdeiley wrote: post #548

People underrate Google's cost effectiveness so much. Half price of Opus. HALF. Think about ANY other product and what you'd expect from the competition thats half the price. Yet people here act like Gemini is dead weight ____ Update: 3.1 was 40% of the cost to run AA index vs Opus Thinking AND SONNET, beat Opus, and still 30% faster for output speed. https://artificialanalysis.ai/?speed=intelligence-vs-speed&m...

We are not at the moment where price matters. All that matters is performance.

What did you say? Cant hear you over the $400B in capex spend.

Counterpoint: price will matter before we hit AGI

Re: Gemini 3.1 Pro

#617

Post by troymc »

ghurtado wrote: post #448
gnatolf wrote: post #399

Earlier quoted context omitted.

So let's put things we're interested in in the benchmarks. I'm not against pelicans!

I think the reason the pelican example is great is because it's bizarre enough that it's unlikely that to appear in the training as one unified picture. If we picked something more common, like say, a hot dog with toppings, then the training contamination is much harder to control.

I think it's now part of their training though, thanks to Simon constantly testing every new model against it, and sharing his results publicly.

There's a specific term for this in education and applied linguistics: the washback effect.

Re: Gemini 3.1 Pro

#618

Post by 306bobby »

metadat wrote: post #591
alphabetting wrote: post #393

Earlier quoted context omitted.

the agentic benchmarks for 3.1 indicate Gemini has caught up. the gains are big from 3.0 to 3.1. For example the APEX-Agents benchmark for long time horizon investment banking, consulting and legal work: 1. Gemini 3.1 Pro - 33.2% 2. Opus 4.6 - 29.8% 3. GPT 5.2 Codex - 27.6% 4. Gemini Flash 3.0 - 24.0% 5. GPT 5.2 - 23.0% 6. Gemini 3.0 Pro - 18.0%

Ranking Codex 5.2 ahead of plain 5.2 doesn't make sense. Codex is expressly designed for coding tasks. Not systems design, not problem analysis, and definitely not banking, but actually solving specific programming tasks (and it's very, very good at this). GPT 5.2 (non-codex) is better in every other way.

It could be problem specific. There are certain non program things that opus seems better than sonnet at as well

Re: Gemini 3.1 Pro

#619

Post by sdeiley »

cyanydeez wrote: post #560
sdeiley wrote: post #548

People underrate Google's cost effectiveness so much. Half price of Opus. HALF. Think about ANY other product and what you'd expect from the competition thats half the price. Yet people here act like Gemini is dead weight ____ Update: 3.1 was 40% of the cost to run AA index vs Opus Thinking AND SONNET, beat Opus, and still 30% faster for output speed. https://artificialanalysis.ai/?speed=intelligence-vs-speed&m...

Some people like blackjack and a technical edge with card counting, others just say screw it and do slot machines.

This is a decent analogy actually. Kudos

Re: Gemini 3.1 Pro

#620

Post by raducu »

s3p wrote: post #325
spankalee wrote: post #199

I hope this works better than 3.0 Pro I'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development. It's stunningly good at reasoning, design, and generating the raw code, but it just falls over a lot when actually trying to get things done, especially compared to Claude Opus. Within VS Code Copi…

Don't get me started on the thinking tokens. Since 2.5P the thinking has been insane. "I'm diving in to the problem", "I'm fully immersed" or "I'm meticulously crafting the answer"

> Don't get me started on the thinking tokens.

Claude provides nicer explanations, but when it comes to CoT tokens or just prompting the LLM to explain -- I'm very skeptical of the truthfulness of it.

Not because the LLM lies, but because humans do that also -- when asked how the figured something, they'll provide a reasonable sounding chain of thought, but it's not how they figured it out.

Post reply on HN