Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

201–210 of 728 posts

Re: Gemini 2.5 Pro Preview

#201

I'm totally lost again! If I use Gemini on the website (gemini.google.com), am I using 2.5 Pro IO edition, or am I using the old one?

http://aistudio.google.com/app/prompts/new_chat?model=gemini...

I get this in AI studio, but does it apply to gemini.google.com?

Re: Gemini 2.5 Pro Preview

#202

Be careful, this model is worse than 03-25 in 10 of the 12 benchmarks (!) I bet they kept training on coding, made everything worse on the way, and tried to hide it under the rug because of the sunk costs.

It seems that trying to build llms is the definition of accepting sunk cost.

Re: Gemini 2.5 Pro Preview

#203

My guess is that they've done a lot of tuning to improve diff based code editing. Gemini 2.5 is fantastic at agentic work, but it still is pretty rough around the edges in terms of generating perfectly matching diffs to edit code. It's probably one of the very few issues with the model. Luckily, aider tracks this. They measure the old gemini 2.5 generating proper diffs 92% of the time. I bet this goes up to ~95-98% h…

What do you mean by agentic work in this context?

Re: Gemini 2.5 Pro Preview

#204
post #11

I don't know if I'm doing something wrong, but every time I ask gemini 2.5 for code it outputs SO MANY comments. An exaggerated amount of comments. Sections comments, step comments, block comments, inline comments, all the gang.

> all the gang

What does that mean?

Re: Gemini 2.5 Pro Preview

#205
post #184

Earlier quoted context omitted.

And a $2.99 drugstore slim wallet calculator with solar power gets it right 100% of the time while billion dollar LLMs can still get arithmetic wrong on occasion.

My hammer can't do any arithmetic at all, why does anyone even use them?

Does it sometimes instead of driving a nail hit random things in the house?

Re: Gemini 2.5 Pro Preview

#206
post #11

I don't know if I'm doing something wrong, but every time I ask gemini 2.5 for code it outputs SO MANY comments. An exaggerated amount of comments. Sections comments, step comments, block comments, inline comments, all the gang.

2.5 was the most impressive model I use, but I agree about the comments. And when refactoring some code it wrote before, it just adds more comments, it becomes like archaeological history (disclaimer: I don’t use it for work, but to see what it can do, so I try to intervene as little as possible, and get it to refactor what it thinks it should)

Re: Gemini 2.5 Pro Preview

#207
post #128
post #74

Interestingly, when compering benchmarks of Experimental 03-25 [1] and Experimental 05-06 [2] it seems the new version scores slightly lower in everything except on LiveCodeBench. [1] https://storage.googleapis.com/model-cards/documents/gemini-... [2] https://deepmind.google/technologies/gemini/

Sounds like they were losing so much money on 2.5-Pro they came up with a forced update that made it cheaper to run. They can't come out with "we've made it worse across the board", nor do they want to be the first to actually raise prices, so instead they made a bit of a distill that's slightly better at coding so they can still spin it positively.

Google doesn't pay the nvidia tax. Their TPUs are designed for Gemini and Gemini designed for their TPUs. Google is no doubt paying far less per token than every other AI house.

Re: Gemini 2.5 Pro Preview

#208

I'm totally lost again! If I use Gemini on the website (gemini.google.com), am I using 2.5 Pro IO edition, or am I using the old one?

Check the dropdown in the top left (on my screen,at least).

Are you referring to gemini.google.com or ai studio? I see 2.5 Pro but is this the right one? I saw a tweet from them saying you have to select Canvas first? I'm so so lost.

Re: Gemini 2.5 Pro Preview

#209

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

You should give it docs for each of your base dependencies in a mcp/tool whatever so it can just consult.

internet also helps.

Also having markdown files with the stack etc and any -rules-

Post reply on HN