Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Improved Gemini 2.5 Flash and Flash-Lite

developers.googleblog.com

Post reply on HN
Print view View original

151–160 of 285 posts

Re: Improved Gemini 2.5 Flash and Flash-Lite

#151

Post by noname120 »

reissbaker wrote: post #146
davidmckayv wrote: post #53

This really captures something I've been experiencing with Gemini lately. The models are genuinely capable when they work properly, but there's this persistent truncation issue that makes them unreliable in practice. I've been running into it consistently, responses that just stop mid-sentence, not because of token limits or content filters, but what appears to be a bug in how the model signals completion. It's been…

FWIW, I think GLM-4.5 or Kimi K2 0905 fit the bill pretty well in terms of complete and consistent. (Disclosure: I'm the founder of Synthetic.new, a company that runs open-source LLMs for monthly subscriptions.)

That’s not a “disclosure”, that’s an ad.

Re: Improved Gemini 2.5 Flash and Flash-Lite

#152

Post by tempest_ »

kridsdale1 wrote: post #131
Al-Khwarizmi wrote: post #111

Earlier quoted context omitted.

I wouldn't call that a nitpick, it's a major annoyance. Version numbers become useless with that kind of policy.

The numbers are branding. The appear to be an indicator of a given year long training run. New “versions” are tweaks of the same base.

Sure and that is why you can call it 2.5.

They just don't want to be pinned down because the shifting sands are useful for the time when the LLM starts to get injected with ads or paid influence.

Re: Improved Gemini 2.5 Flash and Flash-Lite

#153

Post by sally_glance »

kridsdale1 wrote: post #131
Al-Khwarizmi wrote: post #111

Earlier quoted context omitted.

I wouldn't call that a nitpick, it's a major annoyance. Version numbers become useless with that kind of policy.

The numbers are branding. The appear to be an indicator of a given year long training run. New “versions” are tweaks of the same base.

I wish they would actually explain it like that somewhere. Or publish the internal version numbers they must certainly be using to ensure a proper development process.

Re: Improved Gemini 2.5 Flash and Flash-Lite

#154

Post by magicalhippo »

daemonologist wrote: post #24
OGEnthusiast wrote: post #4

I'm not even sure how to evaluate what a "better" LLM is, when I've tried running the exact same model (Qwen3) and prompt and gotten vastly different responses on Qwen Chat vs OpenRouter vs running the model locally.

There several reasons responses from the same model might vary: - "temperature" - intentional random sampling from the most likely next tokens to improve "creativity" and help avoid repetition - quantization - running models with lower numeric precision (saves on both memory and compute, without impacting accuracy too much) - differences in/existence of a system prompt, especially when using something end-user-orient…

AFAIK the batch your query lands in can also matter[1].

Though I imagine this should be a smaller effect than different quantization levels say.

[1]: https://thinkingmachines.ai/blog/defeating-nondeterminism-in...

Re: Improved Gemini 2.5 Flash and Flash-Lite

#155

Post by oasisbob »

ashwindharne wrote: post #27

Google seems to be the main foundation model provider that's really focusing on the latency/TPS/cost dimensions. Anthropic/OpenAI are really making strides in model intelligence, but underneath some critical threshold of performance, the really long thinking times make workflows feel a lot worse in collaboration-style tools, vs a much snappier but slightly less intelligent model. It's a delicate balance, because thes…

> because these Gemini models sometimes feel downright lobotomized compared to claude or gpt-5.

I'm using Gemini (2.5-pro) less and less these days. I used to be really impressived with its deep research capabilities and ability to cite sources reliably.

The last few weeks, it's increasingly argumentative and incapable of recognizing hallucinations around sourcing. I'm tired of arguing with it on basics like RFCs and sources it fabricates, won't validate, and refuses to budge on.

Example prompt I was arguing with it on last night:

> within a github actions workflow, is it possible to get access to the entire secrets map, or enumerate keys in this object?

As recent supply-chain attacks have shown, exfiltrating all the secrets from a Github workflow is as simple as `${{ toJSON(secrets) }}` or `echo ${{ toJSON(secrets) }} | base64` at worse. [1]

Give this prompt a shot! Gemini won't do anything except be obstinately ignorant. With me, it provided a test case workflow, and refused to believe the results. When challenged, expect it to cite unrelated community posts. Chatgpt had no problem with it.

[1] https://github.com/orgs/community/discussions/174045 https://github.com/orgs/community/discussions/47165

Re: Improved Gemini 2.5 Flash and Flash-Lite

#156

Post by Narciss »

mguerville wrote: post #133
golfer wrote: post #109

Earlier quoted context omitted.

Unfortunately Gemini isn't the only culprit here. I've had major problems with ChatGPT reliability myself.

I only hit that problem in voice mode, it'll just stop halfway and restart. It's a jarring reminder of its lack of "real" intelligence

This is more because of VAD - voice activity detection

Re: Improved Gemini 2.5 Flash and Flash-Lite

#157

Post by testycool »

PetrBrzyBrzek wrote: post #38
Liwink wrote: post #5

Gemini 2.5 Flash is an impressive model for its price. However, I don't understand why Gemini 2.0 Flash is still popular. From OpenRouter last week: * xAI: Grok Code Fast 1: 1.15T * Anthropic: Claude Sonnet 4: 586B * Google: Gemini 2.5 Flash: 325B * Sonoma Sky Alpha: 227B * Google: Gemini 2.0 Flash: 187B * DeepSeek: DeepSeek V3.1 (free): 180B * xAI: Grok 4 Fast (free): 158B * OpenAI: GPT-4.1 Mini: 157B * DeepSeek: De…

It’s cheaper and faster. What’s not to understand?

You can get it to be unhinged as well. It's awesome.

Re: Improved Gemini 2.5 Flash and Flash-Lite

#158

Post by istjohn »

oasisbob wrote: post #155
ashwindharne wrote: post #27

Google seems to be the main foundation model provider that's really focusing on the latency/TPS/cost dimensions. Anthropic/OpenAI are really making strides in model intelligence, but underneath some critical threshold of performance, the really long thinking times make workflows feel a lot worse in collaboration-style tools, vs a much snappier but slightly less intelligent model. It's a delicate balance, because thes…

> because these Gemini models sometimes feel downright lobotomized compared to claude or gpt-5. I'm using Gemini (2.5-pro) less and less these days. I used to be really impressived with its deep research capabilities and ability to cite sources reliably. The last few weeks, it's increasingly argumentative and incapable of recognizing hallucinations around sourcing. I'm tired of arguing with it on basics like RFCs and…

You should never argue with an LLM. Adjust the original prompt and rerun it.

Re: Improved Gemini 2.5 Flash and Flash-Lite

#159

Post by oasisbob »

istjohn wrote: post #158
oasisbob wrote: post #155

Earlier quoted context omitted.

> because these Gemini models sometimes feel downright lobotomized compared to claude or gpt-5. I'm using Gemini (2.5-pro) less and less these days. I used to be really impressived with its deep research capabilities and ability to cite sources reliably. The last few weeks, it's increasingly argumentative and incapable of recognizing hallucinations around sourcing. I'm tired of arguing with it on basics like RFCs and…

You should never argue with an LLM. Adjust the original prompt and rerun it.

While arguing may not be productive, I have had good results challenging Gemini on hallucinated sources in the past. eg, "You cited RFC 1918, which is a mistake. Can you try carefully to cite a better source here?" which would get it to re-evaluate, maybe by using another tool, admit the mistake, and allow the research to continue.

With this example, several attempts resulted in the same thing: Gemini expressing a strong belief that Github has a security capability which is really doesn't have.

If someone is able to get Gemini to give an accurate answer to this with a similar question, I'd be very curious to hear what it is.

Re: Improved Gemini 2.5 Flash and Flash-Lite

#160

Post by scosman »

qafy wrote: post #39
scosman wrote: post #2

Ugh. If the model name includes sem_ver version number, increment the version number when making a new release! Anthropic learned this lesson. Google, Deepseek, Kimi, OpenAI and others keep repeating it. This feels like Gemini_2.5_final_FINAL_FINAL_v2.

2.5 isn't the version number, its the model generation. it would only be updated when the underlying model architecture, training, etc are updated. this release is, as the name implies, the same model but likely with hardware optimizations, system prompt, and fine-tuning tweaks applied.

If the weights have changed via training (they have) it’s a new model. This isn’t “hardware optimizations”. It’s additional training/new-weights.
Post reply on HN