Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini-2.5-pro-preview-06-05

deepmind.google

Post reply on HN
Print view View original

31–40 of 237 posts

Re: Gemini-2.5-pro-preview-06-05

#31

Post by op00to »

energy123 wrote: post #16
op00to wrote: post #4

I found Gemini 2.5 Pro highly useful for text summaries, and even reasoning in long conversations... UP TO the last 2 weeks or month. Recently, it seems to totally forget what I'm talking about after 4-5 messages of a paragraph of text each. We're not talking huge amounts of context, but conversational braindeadness. Between ChatGPT's sycophancy, Gemini's forgetfulness and poor attention, I'm just sticking with whate…

I believe it's intentionally nerfed if you use it through the app. Once you use Gemini for a long time you realize they have a number of dark patterns to deter heavy users but maintain the experience for light users. These dark patterns are: - "Something went wrong error" after too many prompts in a day. This was an undocumented rate limit because it never occurs earlier in the day and will immediately disappear if y…

Right! I feel like it will sail through MBs of text data, but remembering what I said two turns ago is just too much.

Re: Gemini-2.5-pro-preview-06-05

#32

Post by abraxas »

I found all the previous Gemini models somewhat inferior even compared to Claude 3.7 Sonnet (and much worse than 4) as my coding assistants. I'm keeping an open mind but also not rushing to try this one until some evaluations roll in. I'm actually baffled that the internet at large seems to be very pumped about Gemini but it's not reflective of my personal experience. Not to be that tinfoil hat guy but I smell at least a bit of astroturf activity around Gemini.

Re: Gemini-2.5-pro-preview-06-05

#33

Post by harrisoned »

op00to wrote: post #4

I found Gemini 2.5 Pro highly useful for text summaries, and even reasoning in long conversations... UP TO the last 2 weeks or month. Recently, it seems to totally forget what I'm talking about after 4-5 messages of a paragraph of text each. We're not talking huge amounts of context, but conversational braindeadness. Between ChatGPT's sycophancy, Gemini's forgetfulness and poor attention, I'm just sticking with whate…

I noticed that same behavior across older Gemini models. I build a chatbot at work around 1.5 Flash, and one day suddenly it was behaving like that. it was perfect before, but after it always saluted the user like it was their first chat, despite me sending the history. And i didn't found any changelog regarding that at the time.

After that i moved to OpenAI, Gemini models just seem unreliable on that regard.

Re: Gemini-2.5-pro-preview-06-05

#34

Post by johnfn »

laweijfmvo wrote: post #28
pelorat wrote: post #18

Why not call it Gemini 2.6?

because the plethora of models and versions is getting ridiculous, and for anyone who's not following LLM news daily, you have no clue what to use. There was never a "Google Search 2.6.4 04-13". You just went to google.com and searched.

Well, Google Search never released an API that millions of people depended on.

Re: Gemini-2.5-pro-preview-06-05

#35

Post by energy123 »

hobofan wrote: post #30
jcuenod wrote: post #5

82.2 on Aider Still actually falling behind the official scores for o3 high. https://aider.chat/docs/leaderboards/

That's the older 05-06 preview, not the new one from today.

They knew that. The 82.2 comes from the new benchmarks in the OP not from the aider url. The aider url was supplied for comparison.

Re: Gemini-2.5-pro-preview-06-05

#37

Post by tempusalaria »

johnfn wrote: post #12

Impressive seeing Google notch up another ~25 ELO on lmarena, on top of the previous #1, which was also Gemini! That being said, I'm starting to doubt the leaderboards as an accurate representation of model ability. While I do think Gemini is a good model, having used both Gemini and Claude Opus 4 extensively in the last couple of weeks I think Opus is in another league entirely. I've been dealing with a number of gn…

I agree I find claude easily the best model, at least for programming which is the only thing I use LLMs for

Re: Gemini-2.5-pro-preview-06-05

#38

Post by bachmeier »

abraxas wrote: post #32

I found all the previous Gemini models somewhat inferior even compared to Claude 3.7 Sonnet (and much worse than 4) as my coding assistants. I'm keeping an open mind but also not rushing to try this one until some evaluations roll in. I'm actually baffled that the internet at large seems to be very pumped about Gemini but it's not reflective of my personal experience. Not to be that tinfoil hat guy but I smell at lea…

> I'm actually baffled that the internet at large seems to be very pumped about Gemini but it's not reflective of my personal experience. Not to be that tinfoil hat guy but I smell at least a bit of astroturf activity around Gemini.

I haven't used Claude, but Gemini has always returned better answers to general questions relative to ChatGPT or Copilot. My impression, which could be wrong, is that Gemini is better in situations that are a substitute for search. How do I do this on the command line, tell me about this product, etc. all give better results, sometimes much better, on Gemini.

Re: Gemini-2.5-pro-preview-06-05

#39

Post by varunneal »

johnfn wrote: post #12

Impressive seeing Google notch up another ~25 ELO on lmarena, on top of the previous #1, which was also Gemini! That being said, I'm starting to doubt the leaderboards as an accurate representation of model ability. While I do think Gemini is a good model, having used both Gemini and Claude Opus 4 extensively in the last couple of weeks I think Opus is in another league entirely. I've been dealing with a number of gn…

Have you tried o3 on those problems? I've found o3 to be much more impressive than Opus 4 for all of my use cases.
Post reply on HN