Gemini-2.5-pro-preview-06-05

I found Gemini 2.5 Pro highly useful for text summaries, and even reasoning in long conversations... UP TO the last 2 weeks or month. Recently, it seems to totally forget what I'm talking about after 4-5 messages of a paragraph of text each. We're not talking huge amounts of context, but conversational braindeadness. Between ChatGPT's sycophancy, Gemini's forgetfulness and poor attention, I'm just sticking with whatever local model du jour fits my needs and whatever crap my company is paying for today. It's super annoying, hopefully Gemini gets its memory back!

Re: Gemini-2.5-pro-preview-06-05

#5

Post by jcuenod » Thu, Jun 05, 2025, 4:56 PM UTC

82.2 on Aider

Still actually falling behind the official scores for o3 high. https://aider.chat/docs/leaderboards/

Re: Gemini-2.5-pro-preview-06-05

#6

Post by vthallam » Thu, Jun 05, 2025, 4:57 PM UTC

As if 3 different preview versions of the same model is not confusing enough, the last two dates are 05-06 and 06-05. They could have held off for a day:)

Re: Gemini-2.5-pro-preview-06-05

#7

Post by unsupp0rted » Thu, Jun 05, 2025, 4:57 PM UTC

Curious to see how this compares to Claude 4 Sonnet in code.

This table seems to indicate it's markedly worse?

https://blog.google/products/gemini/gemini-2-5-pro-latest-pr...

Re: Gemini-2.5-pro-preview-06-05

#8

Post by impulser_ » Thu, Jun 05, 2025, 4:57 PM UTC

unpwn wrote: post #3

I feel like instead of constantly releasing these preview versions with different dates attached they should just add a patch version and bump that.

They can't because if someone has built something around that version they don't want to replace that model with a new model that could provide different results.

Re: Gemini-2.5-pro-preview-06-05

#9

Post by energy123 » Thu, Jun 05, 2025, 4:58 PM UTC

So there's both a 05-06 model and a 06-05 model, and the launch page for 06-05 has some graphs with benchmarks for the 05-06 model but without the 06-05 model?

Re: Gemini-2.5-pro-preview-06-05

#10

Post by nsriv » Thu, Jun 05, 2025, 5:01 PM UTC

impulser_ wrote: post #8

unpwn wrote: post #3

I feel like instead of constantly releasing these preview versions with different dates attached they should just add a patch version and bump that.

They can't because if someone has built something around that version they don't want to replace that model with a new model that could provide different results.

Looking at you Anthropic. 4.0 markedly different from 3.7 in my experience.