Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.1 Pro

blog.google

Post reply on HN
Print view View original

381–390 of 951 posts

Re: Gemini 3.1 Pro

#381

Post by h14h »

embedding-shape wrote: post #73
simonw wrote: post #52

Pretty great pelican: https://simonwillison.net/2026/Feb/19/gemini-31-pro/ - took over 5 minutes though, but I think that's because they're having performance teething problems on launch day.

It's an excellent demonstration of the main issue I have with the Gemini family of models, they always go "above and beyond" to do a lot of stuff, even if I explicitly prompt against it. In this case, most of the SVG ends up consisting not just of a bike and a pelican, but clouds, a sun, a hat on the pelican and so much more. Exactly the same thing happens when you code, it's almost impossible to get Gemini to not do…

Would be really interesting to see an "Eager McBeaver" bench around this concept. When doing real work, a model's ability to stay within the bounds of a given task has almost become more important than its raw capabilities now that every frontier model is so dang good.

Every one of these models is so great at propelling the ship forward, that I increasingly care more and more about which models are the easiest to steer in the direction I actually want to go.

Re: Gemini 3.1 Pro

#382

Post by layer8 »

CamperBob2 wrote: post #329
energy123 wrote: post #321

Earlier quoted context omitted.

Francois Chollet accuses the big labs of targeting the benchmark, yes. It is benchmaxxed.

I don't know what he could mean by that, as the whole idea behind ARC-AGI is to "target the benchmark." Got any links that explain further?

The fact that ARC-AGI has public and semi-private in addition to private datasets might explain it: https://arcprize.org/arc-agi/2/#dataset-structure

Re: Gemini 3.1 Pro

#383

Post by energy123 »

buttered_toast wrote: post #367
1024core wrote: post #260

It got the car wash question perfectly: You are definitely going to have to drive it there—unless you want to put it in neutral and push! While 200 feet is a very short and easy walk, if you walk over there without your car, you won't have anything to wash once you arrive. The car needs to make the trip with you so it can get the soap and water. Since it's basically right next door, it'll be the shortest drive of you…

I think we need to reevaluate what purpose these sorts of questions serve and why they're important in regards to judging intelligence. The model getting it correct or not at any given instance isn't the point, the point is if the model ever gets it wrong we can still assume that it still has some semblance of stochasticity in its output, given that a model is essentially static once it is released. Additionally, hey…

They're important but not as N=1. It's like cherry picking a single question from SimpleQA and going aha! It got it right! Meanwhile it's 8% lower score than some other model when evaluated on all questions.

Re: Gemini 3.1 Pro

#384

Post by Robdel12 »

himata4113 wrote: post #125
Robdel12 wrote: post #86

I really want to use google’s models but they have the classic Google product problem that we all like to complain about. I am legit scared to login and use Gemini CLI because the last time I thought I was using my “free” account allowance via Google workspace. Ended up spending $10 before realizing it was API billing and the UI was so hard to figure out I gave up. I’m sure I can spend 20-40 more mins to sort this ou…

use openrouter instead

This is actually an excellent idea, I’ll give this a shot tonight!

Re: Gemini 3.1 Pro

#385

Post by 0xcb0 »

I'm trying to find the information, is this available on the Gemini CLI script, or is this just the web front-end where I can use this new model?

Re: Gemini 3.1 Pro

#386

Post by tantalor »

jasonjmcghee wrote: post #378
simonw wrote: post #52

Pretty great pelican: https://simonwillison.net/2026/Feb/19/gemini-31-pro/ - took over 5 minutes though, but I think that's because they're having performance teething problems on launch day.

What's crazy is you've influenced them to spend real effort ensuring their model is good at generating animated svgs of animals operating vehicles. The most absurd benchmaxxing. https://x.com/jeffdean/status/2024525132266688757?s=46&t=ZjF...

He's svg-mogging

Re: Gemini 3.1 Pro

#387

Post by XCSme »

XCSme wrote: post #376

Gets 10/10 on my potato benchmarks: https://aibenchy.com/model/google-gemini-3-1-pro-preview-med...

Now I need to write more tests.

It's a bit hard to trick reasoning models, because they explore a lot of the angles of a problem, and they might accidentally have an "a-ha" moment that leads them on the right path. It's a bit like doing random sampling and stumbling upon the right result after doing gradient descent from those points.

Re: Gemini 3.1 Pro

#388

Post by nicce »

jbellis wrote: post #278
spankalee wrote: post #199

I hope this works better than 3.0 Pro I'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development. It's stunningly good at reasoning, design, and generating the raw code, but it just falls over a lot when actually trying to get things done, especially compared to Claude Opus. Within VS Code Copi…

yeah, g3p is as smart or smarter as the other flagships but it's just not reliable enough, it will go into "thinking loops" and burn 10s of 1000s of tokens repeating itself. https://blog.brokk.ai/gemini-3-pro-preview-not-quite-baked/ hopefully 3.1 is better.

> it will go into "thinking loops" and burn 10s of 1000s of tokens repeating itself.

Maybe it is just a genius business strategy.

Re: Gemini 3.1 Pro

#389

Post by kridsdale3 »

alpineman wrote: post #235
Murfalo wrote: post #212

I like to think that all these pelican riding a bicycle comments are unwittingly iteratively creating the optimal cyclist pelican as these comment threads are inevitably incorporated in every training set.

More like half of Google's AI team is hanging out on HN, and they can optimise for that outcome to get a good rep among the dev community.

Hello.

(I'm not aware of anyone doing this, but GDM is quite info-siloed these days, so my lack of knowledge is not evidence it's not happening)

Re: Gemini 3.1 Pro

#390

Post by dist-epoch »

nikcub wrote: post #287
esafak wrote: post #17

Has anyone noticed that models are dropping ever faster, with pressure on companies to make incremental releases to claim the pole position, yet making strides on benchmarks? This is what recursive self-improvement with human support looks like.

and anyone notice that the pace has broken xAI and they were just dropped behind? The frontier improvement release loop is now ant -> openai -> google

Musk said Grok 5 is currently being trained, and it has 7 trillion params (Grok 4 had 3)
Post reply on HN