Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Deep Think

blog.google

Post reply on HN
Print view View original

111–120 of 259 posts

Re: Gemini 2.5 Deep Think

#111

Post by martinald »

danenania wrote: post #103
amelius wrote: post #67

Earlier quoted context omitted.

It could be that your problem was too simple to justify the use of Deep Think. But yes, Google should have figured that out and used a less expensive mode of reasoning.

Model routing is deceptively hard though. It has halting problem characteristics: often only the smartest model is smart enough to accurately determine a task's difficulty. And if you need the smartest model to reliably classify the prompt, it's cheaper to just let it handle the prompt directly. This is why model pickers persist despite no one liking them.

Yes but prompt evaluation is far faster than inference as it can be done (mostly) in parallel, so I don't think that's true.

Re: Gemini 2.5 Deep Think

#112

Post by bitshiftfaced »

highfrequency wrote: post #3

Approach is analogous to Grok 4 Heavy: use multiple "reasoning" agents in parallel and then compare answers before coming back with a single response, taking ~30 minutes. Great results, though it would be more fair for the benchmark comparisons to be against Grok 4 Heavy rather than Grok 4 (the fast, single-agent model).

What makes you sure of that? From the article,

> Deep Think pushes the frontier of thinking capabilities by using parallel thinking techniques. This approach lets Gemini generate many ideas at once and consider them simultaneously, even revising or combining different ideas over time, before arriving at the best answer.

This doesn't exclude the possibility of using multiple agents in parallel, but to me it doesn't necessarily mean that this is what's happening, either.

Re: Gemini 2.5 Deep Think

#113

Post by 827a »

foundry27 wrote: post #34

I started doing some experimentation with this new Deep Think agent, and after five prompts I reached my daily usage limit. For $250 USD/mo that’s what you’ll be getting folks. It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. Anecdotally (from my experience) this was the one feature that enthusiasts in the AI community were interested in to justify the exorbitant price of Google’s Ultra subscription. I…

Similar complaints are happening all over reddit with the Claude Code $200/mo plan and Cursor. The companies with deep VC funding have been subsidizing usage for a year now, but we're starting to see that bleed off.

I think the primary concern of this industry right now is how, relative to the current latest generation models, we simultaneously need intelligence to increase, cost to decrease, effective context windows to increase, and token bandwidths to increase. All four of these things are real bottlenecks to unlocking the "next level" of these tools for software engineering usage.

Google isn't going to make billions on solving advanced math exams.

Re: Gemini 2.5 Deep Think

#114

Post by danenania »

martinald wrote: post #111
danenania wrote: post #103

Earlier quoted context omitted.

Model routing is deceptively hard though. It has halting problem characteristics: often only the smartest model is smart enough to accurately determine a task's difficulty. And if you need the smartest model to reliably classify the prompt, it's cheaper to just let it handle the prompt directly. This is why model pickers persist despite no one liking them.

Yes but prompt evaluation is far faster than inference as it can be done (mostly) in parallel, so I don't think that's true.

The problem is that input token cost dominates output token cost for the majority of tasks.

Once you've given the model your prompt and are reading the first output token for classification, you've already paid most of the cost of just prompting it directly.

That said, there could definitely be exceptions for short prompts where output costs dominate input costs. But these aren't usually the interesting use cases.

Re: Gemini 2.5 Deep Think

#115

Post by irthomasthomas »

dumbmrblah wrote: post #94
irthomasthomas wrote: post #40

You can spin up a version of this at home using simonw's LLM cli with the llm-consortium plugin. Bonus 1: Use any combination of models. Mix n match models from any lab. Bonus 2: Serve your custom consortium on a local API from a single command using the llm-model-gateway plugin and use it in your apps and coding assistants. https://x.com/karpathy/status/1870692546969735361 > uv tool install llm llm install llm-conso…

Thanks! Do you happen to know if there any OpenWebUI plugins similar to this?

You can use this with openwebui already. Just llm install llm-model-gateway. Then after you save a consortium you run llm serve --host 0.0.0.0 This will give you a openai compatible endpoint which you add to your chat client.

Re: Gemini 2.5 Deep Think

#116

Post by HarHarVeryFunny »

dweekly wrote: post #83
highfrequency wrote: post #3

Approach is analogous to Grok 4 Heavy: use multiple "reasoning" agents in parallel and then compare answers before coming back with a single response, taking ~30 minutes. Great results, though it would be more fair for the benchmark comparisons to be against Grok 4 Heavy rather than Grok 4 (the fast, single-agent model).

Dumb (?) question but how is Google's approach here different than Mixture of Experts? Where instead of training different experts to have different model weights you just count on temperature to provide diversity of thought. How much benefit is there in getting the diversity of thought in different runs of the same model versus running a consortium of different model weights and architectures? Is there a paper contr…

MOE is just a way to add more parameters/capacity to a model without making it less efficient to run, since it's done in a way that not all parameters are used for each token passing through the model. The name MOE is a bit misleading since the "experts" are just alternate paths though part of the model, not having any distinct expertise in the way the name might suggest.

Just running the model multiple times on the same input and selecting the best response (according to some judgement) seems a bit of a haphazard way of getting much diversity of response, if that is really all it is doing.

There are multiple alternate approaches to sampling different responses from the model that come to mind, such as:

1) "Tree of thoughts" - generate a partial response (e.g. one token, or one reasoning step), then generate branching continuations of each of those, etc, etc. Compute would go up exponentially according to number of chained steps, unless heavy pruning is done similar to how it is done for MCTS.

2) Separate response planning/brainstorming from response generation by first using a "tree of thoughts" like process just to generate some shallow (e.g. depth < 3) alternate approaches, then use each of those approaches as additional context to generate one or more actual responses (to then evaluate and choose from). Hopefully this would result in some high level variety of response without the cost of of just generating a bunch of responses and hoping that they are usefully diverse.

Re: Gemini 2.5 Deep Think

#117

Post by joshuanapoli »

ccozan wrote: post #95
dweekly wrote: post #78

Earlier quoted context omitted.

"I'm sorry but that wasn't a very interesting question you just asked. I'll spare you the credit and have a cheaper model answer that for you for free. Come back when you have something actually challenging."

Actually why not? Recognizing problem complexity as a fist step is really crucial for such expensive "experts". Humans do the same. And a question to the knowledgeable: does a simple/stupid question cost more in terms of resources then a complex problem? in terms of power consumption.

Just put in the prompt customization to model responses on Marvin from Hitchhiker's Guide.

"Here I am, brain the size of a planet, and they ask me to ..."

Re: Gemini 2.5 Deep Think

#119

Post by Xenoamorphous »

stingraycharles wrote: post #4
highfrequency wrote: post #3

Approach is analogous to Grok 4 Heavy: use multiple "reasoning" agents in parallel and then compare answers before coming back with a single response, taking ~30 minutes. Great results, though it would be more fair for the benchmark comparisons to be against Grok 4 Heavy rather than Grok 4 (the fast, single-agent model).

Yeah the general “discovery” is that using the same reasoning compute effort, but spreading them over multiple different agents generally leads to better results. It solves the “longer thinking leads to worse results” problem by approaching multiple paths of thinking in parallel, but just not think as long.

> Yeah the general “discovery” is that using the same reasoning compute effort, but spreading them over multiple different agents generally leads to better results.

Isn’t the compute effort N times as expensive, where N is the number of agents? Unless you meant in terms of time (and even then, I guess it’d be the slowest of the N agents).

Re: Gemini 2.5 Deep Think

#120

Post by jug »

nickandbro wrote: post #43

Ladies and Gentlemen, Here's Gemini Deep Think when prompted with: "Create a svg of a pelican riding on a bicycle" https://www.svgviewer.dev/s/5R5iTexQ Beat Simon Willison to it :)

Easily the best one yet!
Post reply on HN