Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Deep Think

blog.google

Post reply on HN
Print view View original

141–150 of 259 posts

Re: Gemini 2.5 Deep Think

#141

Post by lucianbr »

Closi wrote: post #124
foundry27 wrote: post #34

I started doing some experimentation with this new Deep Think agent, and after five prompts I reached my daily usage limit. For $250 USD/mo that’s what you’ll be getting folks. It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. Anecdotally (from my experience) this was the one feature that enthusiasts in the AI community were interested in to justify the exorbitant price of Google’s Ultra subscription. I…

It's not particularly interesting if Deep Mind comes to the same (correct) conclusion on a single problem as o3 but costs more. You could ask gpt 2.5 and gpt4 what 1+1= and would get the same response with gpt 4 costing more, but this doesn't tell us much about model capability or value. It would be more interesting to know if it can handle problems that o3 can't do, or if it is 'correct' more often than o3 pro on th…

> It would be more interesting to know if it can handle problems that o3 can't do

Suppose it can't. How will you know? All the datapoints will be "not particularly interesting".

Re: Gemini 2.5 Deep Think

#142

Post by danenania »

redox99 wrote: post #127
danenania wrote: post #114

Earlier quoted context omitted.

The problem is that input token cost dominates output token cost for the majority of tasks. Once you've given the model your prompt and are reading the first output token for classification, you've already paid most of the cost of just prompting it directly. That said, there could definitely be exceptions for short prompts where output costs dominate input costs. But these aren't usually the interesting use cases.

That's usually not the case for thinking models. And usually hard problems have a very short prompt.

For me personally (using mostly for coding and project planning) it's nearly always the case, including with thinking models. I'm usually pasting in a bunch of files, screenshots, etc., and having long conversations. Input nearly always heavily dominates output.

I don't disagree that there are hard problems which use short prompts, like math homework problems etc., but they mostly aren't what I would categorize as "real work". But of course I can only speak to my own experience /shrug.

Re: Gemini 2.5 Deep Think

#143

Post by 827a »

Fade_Dance wrote: post #123
827a wrote: post #113

Earlier quoted context omitted.

Similar complaints are happening all over reddit with the Claude Code $200/mo plan and Cursor. The companies with deep VC funding have been subsidizing usage for a year now, but we're starting to see that bleed off. I think the primary concern of this industry right now is how, relative to the current latest generation models, we simultaneously need intelligence to increase, cost to decrease, effective context window…

Agreed, and big context windows are key to mass adoption in wider use cases beyond chatbots (random ex: in knowledge management apps, being able to parse the entire note library/section and hook it into global AI search), but those use cases are decidedly not areas where $200 per month subscriptions can work. I'll hazard to say that cost and context windows are the two key metrics to bridge that chasm with acceptable…

Big, coherent context windows are key to almost all use-cases. The whole house of cards RAG implementations most platforms are using right now are pretty bad. You start asking around about how to implement RAG and you realize: No one knows, the architecture and outcomes at every company are pretty bad, the most common words you hear are "yeah it pretty much works ok i guess".

Re: Gemini 2.5 Deep Think

#144

Post by danenania »

energy123 wrote: post #129
danenania wrote: post #114

Earlier quoted context omitted.

The problem is that input token cost dominates output token cost for the majority of tasks. Once you've given the model your prompt and are reading the first output token for classification, you've already paid most of the cost of just prompting it directly. That said, there could definitely be exceptions for short prompts where output costs dominate input costs. But these aren't usually the interesting use cases.

No, you're talking about costs to user, which are oversimplifications of the costs that providers bear. One output token with a million input tokens is incredibly cheap for providers

> One output token with a million input tokens is incredibly cheap for providers

Source? Afaik this is incorrect.

Re: Gemini 2.5 Deep Think

#145

Post by NitpickLawyer »

Xenoamorphous wrote: post #119
stingraycharles wrote: post #4

Earlier quoted context omitted.

Yeah the general “discovery” is that using the same reasoning compute effort, but spreading them over multiple different agents generally leads to better results. It solves the “longer thinking leads to worse results” problem by approaching multiple paths of thinking in parallel, but just not think as long.

> Yeah the general “discovery” is that using the same reasoning compute effort, but spreading them over multiple different agents generally leads to better results. Isn’t the compute effort N times as expensive, where N is the number of agents? Unless you meant in terms of time (and even then, I guess it’d be the slowest of the N agents).

Not exactly N times, no. In a traditional transformer arch token 1 is cheaper to generate than token 1000 is cheaper than token 10k and so on. So having 10x 1000 tokens would be cheaper to run concurrently than 10.000 in one session.

You also run into context issues and quality degradation the longer you go.

(this is assuming gemini uses a traditional arch, and not something special regarding attention)

Re: Gemini 2.5 Deep Think

#146

Post by epa »

qingcharles wrote: post #132
LeafItAlone wrote: post #90

Earlier quoted context omitted.

> It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. In my experience Grok 4 and 4 Heavy have been crap. Who cares how many requests you get with it when the response is terrible. Worst LLM money I’ve spent this year and I’ve spent a lot.

It's just wildly inconsistent to me. Some times it'll produce a work of genius. Other times, total garbage.

Unfortunately we are still in the prompt optimization stage, garbage in garbage out

Re: Gemini 2.5 Deep Think

#147

Post by akomtu »

nickandbro wrote: post #43

Ladies and Gentlemen, Here's Gemini Deep Think when prompted with: "Create a svg of a pelican riding on a bicycle" https://www.svgviewer.dev/s/5R5iTexQ Beat Simon Willison to it :)

Now add irrelevant facts about cats and see if it can still draw it.

Re: Gemini 2.5 Deep Think

#148

Post by 827a »

com2kid wrote: post #140
Fade_Dance wrote: post #123

Earlier quoted context omitted.

Agreed, and big context windows are key to mass adoption in wider use cases beyond chatbots (random ex: in knowledge management apps, being able to parse the entire note library/section and hook it into global AI search), but those use cases are decidedly not areas where $200 per month subscriptions can work. I'll hazard to say that cost and context windows are the two key metrics to bridge that chasm with acceptable…

Outside of code, the current RAG strategy is throw shit tons of unstructured text at it that has been found using vector search. Some companies are doing better, but the default rag pipelines are... kind of garbage. For example, a chat bot doing recipe work should have a RAG DB that, by default, returns entire recipes. A vector DB is actually not the solution here, any number of traditional DBs (relational or even a…

IMO: Context engineering is a fascinating topic because it starts approaching the metaphysical abstract idea of what LLMs even are.

If you believe that an LLM is a digital brain, then it follows that their limitation in capabilities today are a result of their limited characteristics (namely: coherent context windows). If we increase context windows (and intelligence), we can simply pack more data into the context, ask specific questions, and let the LLM figure it out.

However, if you have a more grounded belief that, at best, LLMs are just one part of a more heterogeneous digital brain: It follows that maybe actually their limitations are a result of how we're feeding it data. That we need to be smarter about context engineering, we need to do roundtrips with the LLM to narrow down what thbe context should be, it needs targeted context to maximize the quality of its output.

The second situation feels so much harder, but more likely. IMO: This fundamental schism is the single reason why ASI won't be achieved on any timeframe worth making a prediction about. LLMs are just one part of the puzzle.

Re: Gemini 2.5 Deep Think

#149

Post by [deleted] »

WhitneyLand wrote: post #23
cpursley wrote: post #20

I can’t even convince Gemini CLI while planning things to not go off and make a bunch of random changes on its own, even after being very clear not to do so, intercepting to tell it to stop doing that, then it just continues on fucking everything up.

Agents muddy the waters. Claude Code gets the most out of Anthropic’s models, that’s why people love it. Conversely, Gemini CLI makes Gemini Pro 2.5 less capable than the model itself actual is. It’s such a stark difference I’ve given up using Gemini CLI even with it being free, but still use it for situations amenable to a prompt interface on a regular basis. It’s a very strong model.

[deleted]

Re: Gemini 2.5 Deep Think

#150

Post by MaxikCZ »

epa wrote: post #146
qingcharles wrote: post #132

Earlier quoted context omitted.

It's just wildly inconsistent to me. Some times it'll produce a work of genius. Other times, total garbage.

Unfortunately we are still in the prompt optimization stage, garbage in garbage out

I hear this repeated so many times I feel like its a narrative pushed by the sellers. Year ago you could ask for glass of wine filled to the brim and you just wouldnt get it. It wasnt garbage in, garbage out, it was sensibility in, garbage out.

The line where chatbots stop being sensible and start outputting garbage is in movement, but slower than avg joe would guess. You only notice it when you get an intuition of the answer before you see it, which requires a lot of experience on range of complexity. Persisten newbies are the best spotters, because they ask obvious basic questions while asking for stuff beyond what geniuses could solve, and only by getting garbage answer and enduring a process of realizing its actually garbage they truly make wider picture of AI than even most powerusers, who tend to have more balanced querries.

Post reply on HN