Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Flash

developers.googleblog.com

Post reply on HN
Print view View original

411–420 of 582 posts

Re: Gemini 2.5 Flash

#411

Post by int_19h »

deanmoriarty wrote: post #178

Genuine naive question: when it comes to Google HN has generally a negative view of it (pick any random story on Chrome, ads, search, web, working at faang, etc. and this should be obvious from the comments), yet when it comes to AI there is a somewhat notable “cheering effect” for Google to win the AI race that goes beyond a conventional appreciation of a healthy competitive landscape, which may appear as a bit of a…

I dislike Google rather strongly due to their ad-based business model, and I was previously very skeptical of their AI offerings because of very lackluster performance compared to OpenAI and Claude. But I can't help but be impressed with Gemini Pro 2.5 for "deep research" and agentic coding. I have subscriptions with all three so that I can keep up with SOTA, but if I had to choose only one to keep, right now it'd be Gemini.

That said I still don't "cheer" for them and I would really rather someone else win the race. But that is orthogonal to recognition of observed objective superiority.

Re: Gemini 2.5 Flash

#413

Post by thawawaycold »

int_19h wrote: post #404
thawawaycold wrote: post #393

Earlier quoted context omitted.

I hope you're being sarcastic. SPC is necessary because mechanical parts have physical tolerances and manufacturing processes are affected by unavoidable statistical variations; it is beyond idiotic to be provided with a machine that can execute deterministic, repeatable processes and then throw that all into the gutter for mere convenience, justifying that simply because "the time is ripe for SWE to learn statistics…

We don't know how to implement a "deterministic, repeatable process" that can look at a bug in a repo and implement a fix end-to-end.

that is not what OP was talking about though.

Re: Gemini 2.5 Flash

#414

Post by int_19h »

Asraelite wrote: post #407
int_19h wrote: post #403

Earlier quoted context omitted.

My experience has been exactly the opposite - Sonnet did fine on trivial tasks, but couldn't e.g. fix a bug end-to-end (from bug description in the tracker to implementing the fix and adding tests) properly because it couldn't understand how the relevant code worked, whereas Gemini would consistently figure out the root cause and write decent fix & tests. Perhaps this is down to specific tools and their prompts? In m…

> Gemini would consistently figure out the root cause and write decent fix & tests. I feel like you might be using it differently to me. I generally don't ask AI to find the cause of a bug, because it's quite bad at that. I use it to identify relevant parts of the code that could be involved in the bug, and then I come up with my own hypotheses for the cause. Then I use AI to help write tests to validate these hypoth…

I used to use them mostly in "smart code completion" mode myself until very recently. But with all the AI IDEs adding agentic mode, I was curious to see how well that fares if I let it drive.

And we aren't talking about trivial bugs here. For TypeScript, the most impressive bug it handled to date was an async race condition due to missing await causing a property to be overwritten with invalid value. For that one I actually had to do some manual debugging and tell it what I observed, but given that info, it was able to locate the problem in the code all by itself and fix it correctly and come up with a way to test it as well.

For C++, the codebase in question was gdb, the bug was a test issue, and it correctly found problematic code based solely on the test log (but I had to prod it a bit in the right direction for the fix).

I should note that this is Gemini Pro 2.5 specifically. When I tried Google's models previously (for all kinds of tasks), I was very unimpressed - it was noticeably worse than other SOTA models, so I was very skeptical going into this. Indeed, I started with Sonnet precisely because my past experience indicated that it was the best option, and I only tried Gemini after Sonnet fumbled.

Re: Gemini 2.5 Flash

#415

Post by dyauspitr »

Tiberium wrote: post #14
hmaxwell wrote: post #8

I did some testing this morning: Prompt: "can you find any mistakes on my codebase? I put one in there on purpose" + 70,000 tokens of codebase where in one line I have an include for a non-existent file. Claude 3.7 Sonnet (Try 1 - new chat) Claimed it Found a race condition in the api of the admin interface that would be triggered if two admins were to change the room order at the same time. Claude suggested I group…

The thread is about 2.5 Flash though, not 2.5 Pro. Maybe you can try again with 2.5 Flash specifically? Even though it's a small model.

I don’t particularly care about the non frontier models though, I found the comment very useful.

Re: Gemini 2.5 Flash

#416

Post by danpalmer »

tempaccount420 wrote: post #335
simonw wrote: post #293

Earlier quoted context omitted.

The Gemini API runs the Python code for you as part of your single API call, without you having to handle the tool call request yourself.

This is so much cheaper than re-prompting each tool use. I wish this was extended to things like: you could give the model an API endpoint that it can call to execute JS code, and the only requirement is that your API has to respond within 5 seconds (maybe less actually). I wonder if this is what OpenAI is planning to do in the upcoming API update to support tools in o3.

I imagine there wouldn’t bd much of a cost to the provider on the API call there so much longer times may be possible. It’s not like this would hold up the LLM in any way, execution would get suspended while the call is made and the TPU/GPU will serve another request.

Re: Gemini 2.5 Flash

#418

Post by tempthrow »

jmacd wrote: post #250
NoahZuniga wrote: post #125

Earlier quoted context omitted.

The rate limits I've encountered with free api keys has been way lower than the limits advertised.

I agree. I found it unusable for anything but casual usage due to the rate limiting. I wonder if I am just missing something?

I think it's the small TPM limits. I'll be way under the 10-30 requests per minute while using Cline, but it appears that the input tokens count towards the rate limit so I'll find myself limited to one message a minute if I let the conversation go on for too long, ironically due to Gemini's long context window. AFAIK Cline doesn't currently offer an option to limit the context explosion to lower than model capacity.

Re: Gemini 2.5 Flash

#419

Post by Asraelite »

int_19h wrote: post #414
Asraelite wrote: post #407

Earlier quoted context omitted.

> Gemini would consistently figure out the root cause and write decent fix & tests. I feel like you might be using it differently to me. I generally don't ask AI to find the cause of a bug, because it's quite bad at that. I use it to identify relevant parts of the code that could be involved in the bug, and then I come up with my own hypotheses for the cause. Then I use AI to help write tests to validate these hypoth…

I used to use them mostly in "smart code completion" mode myself until very recently. But with all the AI IDEs adding agentic mode, I was curious to see how well that fares if I let it drive. And we aren't talking about trivial bugs here. For TypeScript, the most impressive bug it handled to date was an async race condition due to missing await causing a property to be overwritten with invalid value. For that one I a…

I use it for basically everything I can, not just code completion, including end-to-end bug fixes when it makes sense. But most of the time even the current Gemini and Claude models fail with the hard things.

It might be because most bugs that you would encounter in other languages don't occur in the first place in Rust because of the stronger type system. The race condition one you mentioned wouldn't be possible for example. If something like that would occur, it's a compiler error and the AI fixes it while still in the initial implementation stage by looking at the linter errors. I also put a lot of effort into trying to use coding patterns that do as much validation as possible within the type system. So in the end all that's left are the more difficult bugs where a human is needed to assist (for now at least, I'm confident that the models are only going to get better).

Re: Gemini 2.5 Flash

#420

Post by Alifatisk »

hubraumhugo wrote: post #380

You can get your HN profile analyzed and roasted by it. It's pretty funny :) https://hn-wrapped.kadoa.com/ I'll add a selection for different models soon.

How is this relevant to Gemini 2.5 Flash? I guess it's using it or something?
Post reply on HN