Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Deep Think

blog.google

Post reply on HN
Print view View original

101–110 of 259 posts

Re: Gemini 2.5 Deep Think

#101

Post by declan_roberts »

panarky wrote: post #63
nusl wrote: post #57

Been using Gemini for a few months, somehow it's gotten much, much worse in that time. Hallucinations are very common, and it will argue with you when you point it out. So, don't have much confidence.

In my experience with chat, Flash has gotten much, much better. It's my go-to model even though I'm paying for Pro. Pro is frustrating because it too often won't search to find current information, and just gives stale results from before its training cutoff. Flash doesn't do this much anymore. For coding I use Pro in Gemini CLI. It is amazing at coding, but I'm actually using it more to write design docs, decomp mul…

I like flash because when it's wrong it's wrong very quickly. You can either change the prompt or just solve the problem yourself. It works well for people who can spot the answer as being "wrong"

Re: Gemini 2.5 Deep Think

#103

Post by danenania »

amelius wrote: post #67
foundry27 wrote: post #34

I started doing some experimentation with this new Deep Think agent, and after five prompts I reached my daily usage limit. For $250 USD/mo that’s what you’ll be getting folks. It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. Anecdotally (from my experience) this was the one feature that enthusiasts in the AI community were interested in to justify the exorbitant price of Google’s Ultra subscription. I…

It could be that your problem was too simple to justify the use of Deep Think. But yes, Google should have figured that out and used a less expensive mode of reasoning.

Model routing is deceptively hard though. It has halting problem characteristics: often only the smartest model is smart enough to accurately determine a task's difficulty. And if you need the smartest model to reliably classify the prompt, it's cheaper to just let it handle the prompt directly.

This is why model pickers persist despite no one liking them.

Re: Gemini 2.5 Deep Think

#106

Post by zozbot234 »

littlestymaar wrote: post #37
highfrequency wrote: post #3

Approach is analogous to Grok 4 Heavy: use multiple "reasoning" agents in parallel and then compare answers before coming back with a single response, taking ~30 minutes. Great results, though it would be more fair for the benchmark comparisons to be against Grok 4 Heavy rather than Grok 4 (the fast, single-agent model).

That this kind of approach works is good news for local LLM enthusiasts, as it makes Cloud LLM using this more expensive while local LLM can do so for free up to a point (because LLM inference is limited by memory bandwidth not compute, you can run multiple queries in parallel on your graphic card at the same speed as the single one. Until you become compute-bound of course).

> because LLM inference is limited by memory bandwidth not compute, you can run multiple queries in parallel on your graphic card at the same speed as the single one

I don't think this is correct, especially given MoE. You can save some memory bandwidth by reusing model parameters, but that's about it. It's not giving you the same speed as a single query.

Re: Gemini 2.5 Deep Think

#108

Post by simonw »

nickandbro wrote: post #43

Ladies and Gentlemen, Here's Gemini Deep Think when prompted with: "Create a svg of a pelican riding on a bicycle" https://www.svgviewer.dev/s/5R5iTexQ Beat Simon Willison to it :)

OK that is recognizably a pelican, pretty great!

Re: Gemini 2.5 Deep Think

#109

Post by simonw »

arnaudsm wrote: post #61
nickandbro wrote: post #43

Ladies and Gentlemen, Here's Gemini Deep Think when prompted with: "Create a svg of a pelican riding on a bicycle" https://www.svgviewer.dev/s/5R5iTexQ Beat Simon Willison to it :)

Meme benchmarks like this and Strawberry are funny but very easy to game, I bet they're all over training sets nowadays.

If you train a model on the SVGs of pelicans on a bicycle that are out there already you're going to get a VERY weird looking pelican on a bicycle: https://simonwillison.net/tags/pelican-riding-a-bicycle/
Post reply on HN