Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3.1 Pro

blog.google

Post reply on HN
Print view View original

171–180 of 951 posts

Re: Gemini 3.1 Pro

#171

Post by embedding-shape »

alpineman wrote: post #100
Robdel12 wrote: post #86

I really want to use google’s models but they have the classic Google product problem that we all like to complain about. I am legit scared to login and use Gemini CLI because the last time I thought I was using my “free” account allowance via Google workspace. Ended up spending $10 before realizing it was API billing and the UI was so hard to figure out I gave up. I’m sure I can spend 20-40 more mins to sort this ou…

100% agreed. I wish someone would make a test for how reliably the LLMs follow tool use instructions etc. The pelicans are nice but not useful for me to judge how well a model will slot into a production stack.

At first when I got started with using LLMs I read/analyzed benchmarks, looked at what example prompts people used and so on, but many times, a new model does best at the benchmark, and you think it'll be better, but then in real work, it completely drops the ball. Since then I've stopped even reading benchmarks, I don't care an iota about them, they always seem more misdirected than helpful.

Today I have my own private benchmarks, with tests I run myself, with private test cases I refuse to share publicly. These have been built up during the last 1/1.5 years, whenever I find something that my current model struggles with, then it becomes a new test case to include in the benchmark.

Nowadays it's as easy as `just bench $provider $model` and it runs my benchmarks against it, and I get a score that actually reflects what I use the models for, and it feels like it more or less matches with actually using the models. I recommend people who use LLMs for serious work to try the same approach, and stop relying on public benchmarks that (seemingly) are all gamed by now.

Re: Gemini 3.1 Pro

#172

Post by james2doyle »

sho_hn wrote: post #11
sigmar wrote: post #6

blog post is up- https://blog.google/innovation-and-ai/models-and-research/ge... edit: biggest benchmark changes from 3 pro: arc-agi-2 score went from 31.1% -> 77.1% apex-agents score went from 18.4% -> 33.5%

The touted SVG improvements make me excited for animated pelicans.

The blog post includes a video showcasing the improvements. Looks really impressive: https://blog.google/innovation-and-ai/models-and-research/ge...

Re: Gemini 3.1 Pro

#173

Post by [deleted] »

embedding-shape wrote: post #137
gavinray wrote: post #119

Earlier quoted context omitted.

Do you have Personalization Instructions set up for your LLM models? You can make their responses fairly dry/brief.

I'm mostly using them via my own harnesses, so I have full control of the system prompts and so on. And no matter what I try, Gemini keeps "helpfully" adding code comments every now and then. With every other model, "- Don't add code comments" tends to be enough, but with Gemini I'm not sure how I could stop the comments from eventually appearing.

[deleted]

Re: Gemini 3.1 Pro

#175

Post by PunchTornado »

davidguetta wrote: post #162

Implementation and Sustainability Hardware: Gemini 3 Pro was trained using Google’s Tensor Processing Units (TPUs). TPUs are specically designed to handle the massive computations involved in training LLMs and can speed up training considerably compared to CPUs. TPUs often come with large amounts of high-bandwidth memory, allowing for the handling of large models and batch sizes during training, which can lead to bet…

no. only tpus

Re: Gemini 3.1 Pro

#176

Post by emp17344 »

steve_adams_86 wrote: post #106
simonw wrote: post #52

Pretty great pelican: https://simonwillison.net/2026/Feb/19/gemini-31-pro/ - took over 5 minutes though, but I think that's because they're having performance teething problems on launch day.

Ugh, the gears and chain don't mesh and there's no sprocket on the rear hub But seriously, I can't believe LLMs are able to one-shot a pelican on a bicycle this well. I wouldn't have guessed this was going to emerge as a capability from LLMs 6 years ago. I see why it does now, but... It still amazes me that they're so good at some things.

Is this capability “emergent”, or do AI firms specifically target SVG generation in order to improve it? How would we be able to tell?

Re: Gemini 3.1 Pro

#177

Post by embedding-shape »

WarmWash wrote: post #161
embedding-shape wrote: post #137

Earlier quoted context omitted.

I'm mostly using them via my own harnesses, so I have full control of the system prompts and so on. And no matter what I try, Gemini keeps "helpfully" adding code comments every now and then. With every other model, "- Don't add code comments" tends to be enough, but with Gemini I'm not sure how I could stop the comments from eventually appearing.

I'm pretty sure it writes comments for itself, not for the user. I always let the models comment as much as they want, because I feel it makes the context more robust, especially when cycling contexts often to keep them fresh. There is a tradeoff though, as comments do consumer context. But I tend to pretty liberally dispense of instances and start with a fresh window.

> I'm pretty sure it writes comments for itself, not for the user

Yeah, that sounds worse than "trying to helpful". Read the code instead, why add indirection in that way, just to be able to understand what other models understand without comments?

Re: Gemini 3.1 Pro

#178

Post by mikepurvis »

infthi wrote: post #141
simonw wrote: post #52

Pretty great pelican: https://simonwillison.net/2026/Feb/19/gemini-31-pro/ - took over 5 minutes though, but I think that's because they're having performance teething problems on launch day.

Wonder when will we get something other than a side view

That would be a especially challenging for vector output. I tried just now on ChatGPT 5.2 to jump straight to an image, with this prompt:

"make me a cartoon image of a pelican riding a bicycle, but make it from a front 3/4 view, that is riding toward the viewer."

The result was basically a head-on view, but I expect if you then put that back in and said, "take this image and vectorize it as an SVG" you'd have a much better time than trying to one-shot the SVG directly from a description.

... but of course, if that's so, then what's preventing the model from being smart enough to identify this workflow and follow it on its own to get the task completed?

Re: Gemini 3.1 Pro

#179

Post by gallerdude »

rubicon33 wrote: post #117
onlyrealcuzzo wrote: post #111

We've gone from yearly releases to quarterly releases. If the pace of releases continues to accelerate - by mid 2027 or 2028 we're headed to weekly releases.

But actual progress seems to be slower. These modes are releasing more often but aren’t big leaps.

We used to get one annual release which was 2x as good, now we get quarterly releases which are 25% better. So annually, we’re now at 2.4x better.

Re: Gemini 3.1 Pro

#180

Post by phamilton »

Robdel12 wrote: post #86

I really want to use google’s models but they have the classic Google product problem that we all like to complain about. I am legit scared to login and use Gemini CLI because the last time I thought I was using my “free” account allowance via Google workspace. Ended up spending $10 before realizing it was API billing and the UI was so hard to figure out I gave up. I’m sure I can spend 20-40 more mins to sort this ou…

> For those building with a mix of bash and custom tools, Gemini 3.1 Pro Preview comes with a separate endpoint available via the API called gemini-3.1-pro-preview-customtools. This endpoint is better at prioritizing your custom tools (for example view_file or search_code).

It sounds like there was at least a deliberate attempt to improve it.

Post reply on HN