Gemini 3.1 Pro

Post by holbrad » Fri, Feb 20, 2026, 4:58 PM UTC

maxwellcoffee wrote: post #766

pants2 wrote: post #751

Earlier quoted context omitted.

Strange that you say that because the general consensus (and my experience) seems to be the opposite, as well as the AA-Omniscience Hallucination Rate Benchmark which puts 3.0 Pro among the higher hallucinating models. 3.1 seems to be a noticeable improvement though.

Google actually has the BEST ratings in the AA-Omniscience Index: AA-Omniscience Index (higher is better) measures knowledge reliability and hallucination. It rewards correct answers, penalizes hallucinations, and has no penalty for refusing to answer. Gemini 3.1 is the top spot, followed by 3.0 and then opus 4.6 max

Post by verdverm » Fri, Feb 20, 2026, 5:03 PM UTC

Topfi wrote: post #874

verdverm wrote: post #416

Earlier quoted context omitted.

ChatGPT 4.5 was never released to the public, but it is widely believed to be the foundation the 5.x series is built on. Wonder how GP feels about the minor bumps for other model providers?

Minor version bumps are good and I want model providers to communicate changes. The issue I am having is that Gemini "preview" class models have different deprecation timelines and rate limits, making them impossible to rely on for professional use cases. That's why I'd prefer they finish the 3.0 role out prior to putting resources into deploying a second "preview" class model. For a stable deployment, Google needs a…

Post by sheepscreek » Fri, Feb 20, 2026, 5:15 PM UTC

qnleigh wrote: post #831

sheepscreek wrote: post #580

If it’s any consolation, it was able to one-shot a UI & data sync race condition that even Opus 4.6 struggled to fix (across 3 attempts). So far I like how it’s less verbose than its predecessor. Seems to get to the point quicker too. While it gives me hope, I am going to play it by the ear. Otherwise it’s going to be - Gemini for world knowledge/general intelligence/R&D and Opus/Sonnet 4.6 to finish it off. UPDATE:…

Interesting, I've had similar issues. It seems to be very clumsy when using its internal tooling. I've seen diffs where it accidentally garbled significant amounts of code, which it then had to go in and manually fix. It's also introduced bugs into features that it wasn't supposed to be touching, and when I asked it why it was making changes to I the other code, it answered that it had failed to copy-paste since larg…

Post by moffkalast » Fri, Feb 20, 2026, 6:03 PM UTC

km144 wrote: post #895

moffkalast wrote: post #541

Earlier quoted context omitted.

https://arcprize.org/arc-agi/1/ It's a sort of arbitrary pattern matching thing that can't be trained on in the sense that the MMLU can be, but you can definitely generate billions of examples of this kind of task and train on it, and it will not make the model better on any other task. So in that sense, it absolutely can be. I think it's been harder to solve because it's a visual puzzle, and we know how well today's…

The real question is: Why are people designing benchmarks that, if a model is trained on them, it won't improve the performance of the model at any real-world tasks? Why would anyone care about such benchmarks?

Post by znnajdla » Fri, Feb 20, 2026, 6:42 PM UTC

faebi wrote: post #822

I'm doing Ruby and Gemini 3.0 pro has by far been the best model for me. It writes the nicest ruby code, like I would. Further, it either succeeds or fails hard and obviously. I prefer it failing hard instead of of slowly going weird in my code. Similar in antigravity. Privately it's my absolute favorite. So I'm actually rooting for this.