Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

91–100 of 728 posts

Re: Gemini 2.5 Pro Preview

#92

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

Making LLMs know what they don't know is a hard problem. Many attempts at making them refuse to answer what they don't know caused them to refuse to answer things they did in fact know.

Re: Gemini 2.5 Pro Preview

#93
post #56

Earlier quoted context omitted.

I'm literally working on this particular problem. Locally-run server; browser-based interface instead of TUI/CLI; connects to all the major model APIs; many, many quality of life and feature improvements over other tools that hook into your browser. Drop me a line (see profile) if you're interested in beta testing it when it's out.

I'm actually very happy with everything in Claude code, eg the CLI so im really just curious to try other models

I find that 2.5 Pro has a higher ceiling of understanding, while Claude writes more maintainable code with better comments. If we want to combine them... well, it should be easier to fix 2.5 than Claude. That said, neither is there yet.

Currently Claude Code is a big value-add for Claude. Google has nothing equivalent; aider requires far more manual work.

Re: Gemini 2.5 Pro Preview

#94
post #74

Interestingly, when compering benchmarks of Experimental 03-25 [1] and Experimental 05-06 [2] it seems the new version scores slightly lower in everything except on LiveCodeBench. [1] https://storage.googleapis.com/model-cards/documents/gemini-... [2] https://deepmind.google/technologies/gemini/

This should be the top comment. Cherry-picking is hurting this industry.

I bet they kept training on coding tasks, made everything worse on the way, and tried to hide it under the rug because of the sunk costs.

Re: Gemini 2.5 Pro Preview

#95
post #11

I don't know if I'm doing something wrong, but every time I ask gemini 2.5 for code it outputs SO MANY comments. An exaggerated amount of comments. Sections comments, step comments, block comments, inline comments, all the gang.

It’s annoying, but I’ve done extensive work with this model and leaving the comments in for the first few iterations produced better outcomes. I expect this is baked into the RL they’re doing, but because of the context size, it’s not really an issue. You can just ask it to strip out in the final pass.

Re: Gemini 2.5 Pro Preview

#97
post #57
post #11

I don't know if I'm doing something wrong, but every time I ask gemini 2.5 for code it outputs SO MANY comments. An exaggerated amount of comments. Sections comments, step comments, block comments, inline comments, all the gang.

I have the same issue plus unnecessary refactorings (that break functionality). it doesn't matter if I write a whole paragraph in the chat or the prompt explaining I don't want it to change anything else apart from what is required to fulfill my very specific request. It will just go rogue and massacre the entirety of the file.

This is generally controllable with prompting. I usually include something like, “be excessively cautious and conservative in refactoring, only implementing the desired changes” to avoid.

Re: Gemini 2.5 Pro Preview

#98

I don't understand what I'm doing wrong.. it seems like everyone is saying Gemini is better, but I've compared dozens of examples from my work, and Grok has always produced better results.

I haven't tested this release yet, but I found Gemini to be overrated before.

My choice of LLMs was

Coding in cursor: Claude

General questions: Grok, if it fails then Gemini

Deep Research: Gemini (I don't have GPT plus, I heard it's better)

Re: Gemini 2.5 Pro Preview

#99
post #4

>Best-in-class frontend web development It really is wild to have seen this happen over the last year. The days of traditional "design-to-code" FE work are completely over. I haven't written a line of HTML/CSS in months. If you are still doing this stuff by hand, you need to adapt fast. In conjunction with an agentic coding IDE and a few MCP tools, weeks worth of UI work are now done in hours to a higher level of qua…

If it's zero effort, then why do devs need to adapt fast? And wouldn't adapting be incredibly easy?

The only disadvantage to not using these tools would be that your current output is slower. As soon as your employer asks for more or you're looking for a new job, you can just turn on AI and be as fast as everyone who already uses it.

Re: Gemini 2.5 Pro Preview

#100
post #72
post #57

Earlier quoted context omitted.

I have the same issue plus unnecessary refactorings (that break functionality). it doesn't matter if I write a whole paragraph in the chat or the prompt explaining I don't want it to change anything else apart from what is required to fulfill my very specific request. It will just go rogue and massacre the entirety of the file.

This has also been my biggest gripe with Gemini 2.5 Pro. While it is fantastic at one-shotting major new features, when wanting to make smaller iterative changes, it always does big refactors at the same time. I haven't found a way to change that behavior through changes in my prompts. Claude 3.7 Sonnet is much more restrained and does smaller changes.

This exact problem is something I’m hoping to fix with a tool that parses the source to AST and then has the LLM write code to modify the AST (which you then run to get your changes) rather than output code directly.

I’ve started in a narrow niche of python/flask webapps and constrained to that stack for now, but if you’re interested I’ve just opened it for signups: https://codeplusequalsai.com

Would love feedback! Especially if you see promising results in not getting huge refactors out of small change requests!

(Edit: I also blogged about how the AST idea works in case you're just that curious: https://codeplusequalsai.com/static/blog/prompting_llms_to_m...)

Post reply on HN