Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Pro Preview

developers.googleblog.com

Post reply on HN
Print view View original

211–220 of 728 posts

Re: Gemini 2.5 Pro Preview

#211

Post by neilellis »

Tainnor wrote: post #191
prpl wrote: post #52

Earlier quoted context omitted.

Production quality code is defensive. Probably trained on a lot of google code.

Depends on what you mean by "defensive". Anticipating error and non-happy-path cases and handling them is definitely good. Also fault tolerance, i.e. allowing parts of the application to fail without bringing down everything. But I've heard "defensive code" used for the kind of code where almost every method validates its input parameters, wraps everything in a try-catch, returns nonsensical default values in failure…

this!

lots of hasattr("") rubbish, I've increased the amount of prompting but it still does this - basically it defers it's lack of compile time knowledge to runtime 'let's hope for the best, and see what happens!'

Trying to teach it FAIL FAST is an uphill struggle.

Oh and yes, returning mock objects if something goes wrong is a favourite.

It truly is an Idiot Savant - but still amazingly productive.

Re: Gemini 2.5 Pro Preview

#212

Post by HenriNext »

cryptoz wrote: post #100
mgw wrote: post #72

Earlier quoted context omitted.

This has also been my biggest gripe with Gemini 2.5 Pro. While it is fantastic at one-shotting major new features, when wanting to make smaller iterative changes, it always does big refactors at the same time. I haven't found a way to change that behavior through changes in my prompts. Claude 3.7 Sonnet is much more restrained and does smaller changes.

This exact problem is something I’m hoping to fix with a tool that parses the source to AST and then has the LLM write code to modify the AST (which you then run to get your changes) rather than output code directly. I’ve started in a narrow niche of python/flask webapps and constrained to that stack for now, but if you’re interested I’ve just opened it for signups: https://codeplusequalsai.com Would love feedback! E…

Interesting idea. But LLMs are trained on vast amount of "code as text" and tiny fraction of "code as AST"; wouldn't that significantly hurt the result quality?

Re: Gemini 2.5 Pro Preview

#213

Post by tough »

diggan wrote: post #117
brandall10 wrote: post #68

Earlier quoted context omitted.

It's certainly annoying, but you can try following up with "can you please remove superfluous comments? In particular, if a comment doesn't add anything to the understanding of the code, it doesn't deserve to be there".

I'm having the same issue, and no matter what I prompt (even stuff like "Don't add any comments at all to anything, at any time") it still tries to add these typical junior-dev comments where it's just re-iterating what the code on the next line does.

you can have a script that drops them all

Re: Gemini 2.5 Pro Preview

#214

Post by m_kos »

[Tangent] Anyone here using 2.5 Pro in Gemini Advanced? I have been experiencing a ton of bugs, e.g.,:

- [codes] showing up instead of references,

- raw search tool output sliding across the screen,

- Gemini continusly answering questions asked two or more messages before but ignoring the most recent one (you need to ask Gemini an unrelated question for it to snap out of this bug for a few minutes),

- weird messages including text irrelevant to any of my chats with Gemini, like baseball,

- confusing its own replies with mine,

- not being able to run its own Python code due to some unsolvable formatting issue,

- timeouts, and more.

Re: Gemini 2.5 Pro Preview

#215

Post by tough »

cryptoz wrote: post #100
mgw wrote: post #72

Earlier quoted context omitted.

This has also been my biggest gripe with Gemini 2.5 Pro. While it is fantastic at one-shotting major new features, when wanting to make smaller iterative changes, it always does big refactors at the same time. I haven't found a way to change that behavior through changes in my prompts. Claude 3.7 Sonnet is much more restrained and does smaller changes.

This exact problem is something I’m hoping to fix with a tool that parses the source to AST and then has the LLM write code to modify the AST (which you then run to get your changes) rather than output code directly. I’ve started in a narrow niche of python/flask webapps and constrained to that stack for now, but if you’re interested I’ve just opened it for signups: https://codeplusequalsai.com Would love feedback! E…

Interesting, i started playing with ts-morph and neo4j to parse TypeScript codebases.

simonw has symbex which could be useful for you for python

Re: Gemini 2.5 Pro Preview

#216

Post by jjmarr »

ctxc wrote: post #182
pizza wrote: post #174

Earlier quoted context omitted.

Are you sure about that? Try these.. - (1e(1e10) + 1) - 1e(1e10) - sqrt(sqrt(2)) * sqrt(sqrt(2)) * sqrt(sqrt(2)) * sqrt(sqrt(2))

Three decades and I haven't had to do anything remotely resembling this on a calculator, much less find the calculator wrong. Same for the majority of general population I assume.

(1/3)*3

Re: Gemini 2.5 Pro Preview

#217

Post by siscia »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

This problem have been solved by LSP (language server protocol), all we need is a small server behind MCP that can communicate LSP information back to the LLM and get the LLM to use by adding to the prompt something like: "check your API usage with the LSP"

The unfortunate state of open source funding makes buildings such simple tool a loosing adventure unfortunately.

Re: Gemini 2.5 Pro Preview

#218

Post by kenjackson »

maccard wrote: post #186
jerkstate wrote: post #139

Earlier quoted context omitted.

There are a bunch of stupid behaviors of LLM coding that will be fixed by more awareness pretty soon. Imagine putting the docs and code for all of your libraries into the context window so it can understand what exceptions might be thrown!

Copilot and the likes have been around for 4 years, and we’ve been hearing this all along. I’m bullish on LLM assistants (not vibe coding) but I’d love to see some of these things actually start to happen.

I feel like it has gotten better over time, but I don't have any metrics to confirm this. And it may also depend on what type of you language/libraries that you use.

Re: Gemini 2.5 Pro Preview

#219

Post by jug »

segphault wrote: post #18

My frustration with using these models for programming in the past has largely been around their tendency to hallucinate APIs that simply don't exist. The Gemini 2.5 models, both pro and flash, seem significantly less susceptible to this than any other model I've tried. There are still significant limitations, no amount of prompting will get current models to approach abstraction and architecture the way a person doe…

I’ve seen benchs on hallucinations and OpenAI has typically performed worse than Google and Anthropic models. Sometimes significantly so. But it doesn’t seem like they have cared much. I’ve suspected that LLM performance is correlated to risking hallucinations? That is, if they’re bolder, this can be beneficial? Which helps in other performance benchmarks. But of course at the risk of hallucinating more…
Post reply on HN