Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

Post reply on HN
Print view View original

271–280 of 728 posts

Re: Gemini 2.5 Pro Preview

#271

Post by laborcontract »

laborcontract wrote: post #79

My guess is that they've done a lot of tuning to improve diff based code editing. Gemini 2.5 is fantastic at agentic work, but it still is pretty rough around the edges in terms of generating perfectly matching diffs to edit code. It's probably one of the very few issues with the model. Luckily, aider tracks this. They measure the old gemini 2.5 generating proper diffs 92% of the time. I bet this goes up to ~95-98% h…

Update 2: I've been using this model in both aider and cline and I've haven't gotten a diff matching error yet, even with some pretty difficult substitutions across different places in multiple files. The overall feel of this model is nice.

I don't have a formal benchmark but there's a notable improvement in code generation due to this alone.

I've had gemini chug away on plans that have taken ~1 hour to implement. (~80mln tokens spent) A good portion of that energy was spent fixing mistakes made by cline/aider/roo due to search/replace mistakes. If this model gets anywhere close to 100% on diffs then this is a BFD. I estimate this will translate to a 50-75% productivity boost on long context coding tasks. I hope the initial results i'm seeing hold up!

I'm surprised by the reaction in the rest of the thread. A lot unproductive complaining, a lot of off topic stuff, nothing talking about the model itself.

Any thoughts from anyone else using the updated model?

Re: Gemini 2.5 Pro Preview

#272

Post by dotancohen »

Jordan-117 wrote: post #247
segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

I recently needed to recommend some IAM permissions for an assistant on a hobby project; not complete access but just enough to do what was required. Was rusty with the console and didn't have direct access to it at the time, but figured it was a solid use case for LLMs since AWS is so ubiquitous and well-documented. I actually queried 4o, 3.7 Sonnet, and Gemini 2.5 for recommendations, stripped the list of duplicate…

AWS docs have (had) an embedded AI model that would do this perfectly. I suppose it had better training data, and the actual spec as a RAG.

Re: Gemini 2.5 Pro Preview

#273

Post by freddydumont »

ranyume wrote: post #11

I don't know if I'm doing something wrong, but every time I ask gemini 2.5 for code it outputs SO MANY comments. An exaggerated amount of comments. Sections comments, step comments, block comments, inline comments, all the gang.

That’s been my experience as well. It’s especially jarring when asking for a refactor as it will leave a bunch of WIP-style comments highlighting the difference with the previous approach.

Re: Gemini 2.5 Pro Preview

#274

Post by fwip »

mynameisvlad wrote: post #240
ajross wrote: post #230

Earlier quoted context omitted.

> Are we sure they know these things as opposed to being able to consistently guess correctly? What is the practical difference you're imagining between "consistently correct guess" and "knowledge"? LLMs aren't databases. We have databases. LLMs are probabilistic inference engines. All they do is guess, essentially. The discussion here is about how to get the guess to "check itself" with a firmer idea of "truth". And…

Simple, and even simpler from your own example. Knowledge has an objective correctness. We know that there is a "right" and "wrong" answer and we know what a "right" answer is. "Consistently correct guesses", based on the name itself, is not reliable enough to actually be trusted. There's absolutely no guarantee that the next "consistently correct guess" is knowledge or a hallucination.

So, if that were so, then an LLM possess no knowledge whatsoever, and cannot ever be trusted. Is that the line of thought you are drawing?

Re: Gemini 2.5 Pro Preview

#275

Post by satvikpendem »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

If you use Cursor, you can use @Docs to let it index the documentation for the libraries and languages you use, so no hallucination happens.

Re: Gemini 2.5 Pro Preview

#276

Post by fwip »

ChocolateGod wrote: post #83
segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

I asked today both Claude and ChatGPT to fix a Grafana Loki query I was trying to build, both hallucinated functions that didn't exist, even when telling to use existing functions. To my surprise, Gemini got it spot on first time.

Could be a bit of a "it's always in the last place you look" kind of thing - if Claude or CGPT had gotten it right, you wouldn't have tried Gemini.

Re: Gemini 2.5 Pro Preview

#277

Post by qwertox »

I have my issues with the code Gemini Pro in AI Studio generates without customized "System Instructions".

It turns a well readable code-snippet of 5 lines into a 30 line snippet full of comments and mostly unnecessary error handling. Code which becomes harder to reason about.

But for sysadmin tasks, like dealing with ZFS and LVM, it is absolutely incredible.

Re: Gemini 2.5 Pro Preview

#278

Post by throwup238 »

taf2 wrote: post #38
ranyume wrote: post #11

I don't know if I'm doing something wrong, but every time I ask gemini 2.5 for code it outputs SO MANY comments. An exaggerated amount of comments. Sections comments, step comments, block comments, inline comments, all the gang.

I really liked the Gemini 2.5 pro model when it was first released - the upload code folder was very nice (but they removed it). The annoying things I find with the model is it does a really bad job of formatting the code it generates... I know I can use a code formatting tool and I do when i use gemini output but otherwise I find grok much easier to work with and yields better results.

> I really liked the Gemini 2.5 pro model when it was first released - the upload code folder was very nice (but they removed it).

Removed from where? I use the attach code folder feature every day from the Gemini web app (with a script that clones a local repo that deletes .git and anything matching a gitignore pattern).

Re: Gemini 2.5 Pro Preview

#279

Post by martinsnow »

paulirish wrote: post #250

> Gemini 2.5 Pro now ranks #1 on the WebDev Arena leaderboard It'd make sense to rename WebDev Arena to React/Tailwind Arena. Its system prompt requires [1] those technologies and the entire tool breaks when requesting vanilla JS or other frameworks. The second-order implications of models competing on this narrow definition of webdev are rather troublesome. [1] https://blog.lmarena.ai/blog/2025/webdev-arena/#:~:text…

Bwoah it's almost as if react and tailwind is the bees knees ind frontend atm
Post reply on HN