Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.1 Pro

blog.google

Post reply on HN
Print view View original

521–530 of 951 posts

Re: Gemini 3.1 Pro

#521

Post by renegade-otter »

karmasimida wrote: post #362
spankalee wrote: post #199

I hope this works better than 3.0 Pro I'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development. It's stunningly good at reasoning, design, and generating the raw code, but it just falls over a lot when actually trying to get things done, especially compared to Claude Opus. Within VS Code Copi…

Gemini just doesn’t do even mildly well in agentic stuff and I don’t know why. OpenAI has mostly caught up with Claude in agentic stuff, but Google needs to be there and be there quickly

It's like anything Google - they do the cool part and then lose interest with the last 10%. Writing code is easy, building products that print money is hard.

Re: Gemini 3.1 Pro

#523

Post by jmkni »

1024core wrote: post #331
1024core wrote: post #260

It got the car wash question perfectly: You are definitely going to have to drive it there—unless you want to put it in neutral and push! While 200 feet is a very short and easy walk, if you walk over there without your car, you won't have anything to wash once you arrive. The car needs to make the trip with you so it can get the soap and water. Since it's basically right next door, it'll be the shortest drive of you…

Some people are suggesting that this might actually be in the training set. Since I can't rule that out, I tried a different version of the question, with an elephant instead of a car: > It's a hot and dusty day in Arizona and I need to wash my elephant. There's a creek 300 feet away. Should I ride my elephant there or should I just walk there by myself? Gemini said: That sounds like quite the dusty predicament! Give…

I like telling it the car wash is 24,901.1 miles away (with the implication being that I'm right beside the car wash)

Re: Gemini 3.1 Pro

#524

Post by z3t4 »

casey2 wrote: post #486
emp17344 wrote: post #29

Earlier quoted context omitted.

Remember when ARC 1 was basically solved, and then ARC 2 (which is even easier for humans) came out, and all of the sudden the same models that were doing well on ARC 1 couldn’t even get 5% on ARC 2? Not convinced these benchmark improvements aren’t data leakage.

ARC 2 was made specifically to artificially lower contemporary LLM scores, therefore any kind of model improvements will have outsized effects Also people use "saturated" too liberally. The top left corner 1 cent per task is saturated IMO. Since there are billions of people who would perfer to solve arc 1 tasks at 52 cents per task. Arc 2 a human would make thousands of dollars a day with 99.99% accuracy

How much do I get if I solve this? :D

https://arcprize.org/play

Re: Gemini 3.1 Pro

#525

Post by blueaquilae »

alphabetting wrote: post #393
karmasimida wrote: post #362

Earlier quoted context omitted.

Gemini just doesn’t do even mildly well in agentic stuff and I don’t know why. OpenAI has mostly caught up with Claude in agentic stuff, but Google needs to be there and be there quickly

the agentic benchmarks for 3.1 indicate Gemini has caught up. the gains are big from 3.0 to 3.1. For example the APEX-Agents benchmark for long time horizon investment banking, consulting and legal work: 1. Gemini 3.1 Pro - 33.2% 2. Opus 4.6 - 29.8% 3. GPT 5.2 Codex - 27.6% 4. Gemini Flash 3.0 - 24.0% 5. GPT 5.2 - 23.0% 6. Gemini 3.0 Pro - 18.0%

Marketing team agree with benchmark score...

Re: Gemini 3.1 Pro

#526

Post by brikym »

simonw wrote: post #52

Pretty great pelican: https://simonwillison.net/2026/Feb/19/gemini-31-pro/ - took over 5 minutes though, but I think that's because they're having performance teething problems on launch day.

Another great benchmark would be to convert a raster image of a logo into SVG. I've yet to find a good tool for this that produces accurate smooth lines.

Re: Gemini 3.1 Pro

#527

Post by menaerus »

stephen_cagle wrote: post #470
spankalee wrote: post #199

I hope this works better than 3.0 Pro I'm a former Googler and know some people near the team, so I mildly root for them to at least do well, but Gemini is consistently the most frustrating model I've used for development. It's stunningly good at reasoning, design, and generating the raw code, but it just falls over a lot when actually trying to get things done, especially compared to Claude Opus. Within VS Code Copi…

I also worked at Google (on the original Gemini, when it was still Bard internally) and my experience largely mirrors this. My finding is that Gemini is pretty great for factual information and also it is the only one that I can reliably (even with the video camera) take a picture of a bird and have it tell me what the bird is. But it is just pretty bad as a model to help with development, myself and everyone I know…

I don't know ... as of now I am literally instructing it to solve the chained expression computation problem which incurs a lot of temporary variables, of which some can be elided by the compiler and some cannot. Think linear algebra expressions which yield a lot of intermediate computations for which you don't want to create a temporary. This is production code and not an easy problem.

And yet it happily told me what I exactly wanted it to tell me - rewrite the goddamn thing using the (C++) expression templates. And voila, it took "it" 10 minutes to spit out the high-quality code that works.

My biggest gripe for now with Gemini is that Antigravity seems to be written by the model and I am experiencing more hiccups than I would like to, sometimes it's just stuck.

Re: Gemini 3.1 Pro

#528

Post by ertgbnm »

nickandbro wrote: post #51

Does well on SVGs outside of "pelican riding on a bicycle" test. Like this prompt: "create a svg of a unicorn playing xbox" https://www.svgviewer.dev/s/NeKACuHj Still some tweaks to the final result, but I am guessing with the ARC-AGI benchmark jumping so much, the model's visual abilities are allowing it to do this well.

Animated SVGs are one of the example in the press release. Which is fine, I just think the weird SVG benchmark is now dead. Gemini has beat the benchmark and now differences are just coming down to taste.

I don't know if it got these abilities through generalization or if google gave it a dedicated animated SVG RL suite that got it to improve so much between models.

Regardless we need a new vibe check benchmark ala bicycle pelican.

Re: Gemini 3.1 Pro

#529

Post by girvo »

alphabetting wrote: post #393
karmasimida wrote: post #362

Earlier quoted context omitted.

Gemini just doesn’t do even mildly well in agentic stuff and I don’t know why. OpenAI has mostly caught up with Claude in agentic stuff, but Google needs to be there and be there quickly

the agentic benchmarks for 3.1 indicate Gemini has caught up. the gains are big from 3.0 to 3.1. For example the APEX-Agents benchmark for long time horizon investment banking, consulting and legal work: 1. Gemini 3.1 Pro - 33.2% 2. Opus 4.6 - 29.8% 3. GPT 5.2 Codex - 27.6% 4. Gemini Flash 3.0 - 24.0% 5. GPT 5.2 - 23.0% 6. Gemini 3.0 Pro - 18.0%

Benchmarks are basically straight up meaningless at this point in my experience. If they mattered and were the whole story, those Chinese open models would be stomping the competition right now. Instead they're merely decent when you use them in anger for real work.

I'll withhold judgement until I've tried to use it.

Re: Gemini 3.1 Pro

#530

Post by logicchains »

mijoharas wrote: post #205

Gemini 3 is still in preview (limited rate limits) and 2.5 is deprecated (still live but won't be for long).[0] Are Google planning to put any of their models into production any time soon? Also somewhat funny that some models are deprecated without a suggested alternative(gemini-2.5-flash-lite). Do they suggest people switch to Claude? [0] https://ai.google.dev/gemini-api/docs/deprecations

It's the Tensorflow strategy; deprecate the old version while the new version still doesn't support all the old one's functionality.
Post reply on HN