Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Deep Think

blog.google

Post reply on HN
Print view View original

191–200 of 259 posts

Re: Gemini 2.5 Deep Think

#191

Post by robwwilliams »

nyrikki wrote: post #134
ccozan wrote: post #95

Earlier quoted context omitted.

Actually why not? Recognizing problem complexity as a fist step is really crucial for such expensive "experts". Humans do the same. And a question to the knowledgeable: does a simple/stupid question cost more in terms of resources then a complex problem? in terms of power consumption.

IIRC that isn't possible under current models at least in general, for multiple reasons, including attention cannot attend to future tokens, the fact that they are existential logic, that they are really NLP and not NLU, etc... Even proof mining and the Harrop formula have to exclude disjunction and existential quantification to stay away from intuitionist math. IID in PAC/ML implies PEM which is also intentionally e…

Hmm, I needed Claude 4’s help to parse your response. The critique was not too kind to your abbreviated arguments that current systems are not able to gauge the complexity of a prompt and the resources needed to address a question.

Re: Gemini 2.5 Deep Think

#192

Post by espadrine »

lostmsu wrote: post #160
qingcharles wrote: post #135

Earlier quoted context omitted.

This feels like the best pelicanbike yet. The singularity might be closer than we imagine. Time for a leaderboard?

Ask and you'll receive: https://pelicans.borg.games/

It would be interesting to have two generations per model without cherry picking, so that the Elo estimation can include an easy-to-compute standard deviation estimation.

Re: Gemini 2.5 Deep Think

#193

Post by com2kid »

827a wrote: post #148
com2kid wrote: post #140

Earlier quoted context omitted.

Outside of code, the current RAG strategy is throw shit tons of unstructured text at it that has been found using vector search. Some companies are doing better, but the default rag pipelines are... kind of garbage. For example, a chat bot doing recipe work should have a RAG DB that, by default, returns entire recipes. A vector DB is actually not the solution here, any number of traditional DBs (relational or even a…

IMO: Context engineering is a fascinating topic because it starts approaching the metaphysical abstract idea of what LLMs even are. If you believe that an LLM is a digital brain, then it follows that their limitation in capabilities today are a result of their limited characteristics (namely: coherent context windows). If we increase context windows (and intelligence), we can simply pack more data into the context, a…

Information in an LLM exists in two places:

1. Embedded in the parameters

2. Within the context window

We all talk a lot about #2, but until we get a really good grip on #1, I think we as a field are going to hit a progress wall.

The problem is we have not been able to separate out knowledge embedded in parameters with model capability, famously even if you don't want a model to write code, throwing a bunch of code at a model makes it a better model. (Also famously, even if someone never grows up to work with math day to day, learning math makes them better at all sorts of related logical thinking tasks.)

Also there is plenty of research showing performance degrades as we stuff more and more into context. This is why even the best models have limits on tool call performance when naively throwing 15+ JSON schemas at it. (The technique to use RAG to determine which tool call schema to feed into the context window is super cool!)

Re: Gemini 2.5 Deep Think

#194

Post by com2kid »

Fade_Dance wrote: post #186
827a wrote: post #148

Earlier quoted context omitted.

IMO: Context engineering is a fascinating topic because it starts approaching the metaphysical abstract idea of what LLMs even are. If you believe that an LLM is a digital brain, then it follows that their limitation in capabilities today are a result of their limited characteristics (namely: coherent context windows). If we increase context windows (and intelligence), we can simply pack more data into the context, a…

It's also a question of general vs specialized tools. If LLMs are being used in a limited capacity, such as retrieving recipes, then a limited environment where it only has the ability to retrieve complete recipes via RAG may be ideal in the literal sense of the word. There really is nothing better than the perfect specialized tool for a specialized job.

I did embedded work for years. A 100mhz CPU with 1 cycle SRAM latency and a bare metal OS can do as much as a 600MHZ CPU hitting DRAM running a preemptive OS.

Specialized tools rock.

Re: Gemini 2.5 Deep Think

#195

Post by nxobject »

danenania wrote: post #100
LeafItAlone wrote: post #90

Earlier quoted context omitted.

> It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. In my experience Grok 4 and 4 Heavy have been crap. Who cares how many requests you get with it when the response is terrible. Worst LLM money I’ve spent this year and I’ve spent a lot.

It's interesting how multi-dimensional LLM capabilities have proven to be. OpenAI reasoning models (o1-pro, o3, o3-pro) have been the strongest, in my experience, at harder problems, like finding race conditions in intricate concurrency code, yet they still lag behind even the initial sonnet 3.5 release for writing basic usable code. The OpenAI models are kind of like CS grads who can solve complex math problems but…

Those may have been the exact people creating training material for OpenAI…

Re: Gemini 2.5 Deep Think

#196

Post by hirako2000 »

red75prime wrote: post #188
Melatonic wrote: post #177

Earlier quoted context omitted.

This has been my thought for a long time - unless there is some breakthrough in AI algo I feel like we are going to hit a "creativity wall" for coding (and some other tasks).

Any reason to think that the wall will be under the human level?

Off the thousands of responses I have read from the top LLMs in the last couple of years: never seen one that was creative. Throwing writing, coding, problem solving, mathematical questions and what not.

It's somewhat easier to perceive the creativeless aspect with stable diffusion. I'm not talking about the missing limb or extra finger glitches. With a bit of experience looking through generated images our brain eventually perceives the absolute lack of creativity, an artist probably spot it without prior experience with generative AI pieces. With LLMs it takes a bit longer.

Anecdotal, baseless I guess. Papers were published, some researchers in the fields of science couldn't get the best LLMs to solve any unsolved problem. I recently came across a paper stating bluntly that all LLMs tested were unable to conceptualize, nor derive laws that generalize whatsoever. E.g formulas.

We are being duped, it doesn't help selling $200 monthly subscriptions - soon for even more - if marketers admitted there is absolutely zero reasoning going on with these stochastic machines on steroids.

I deeply wish the circus ends soon, so that we can start focusing on what LLMs are excellent, well fitted to do better, faster than humans.

Creative it is not.

Re: Gemini 2.5 Deep Think

#197

Post by hirako2000 »

thimabi wrote: post #42
foundry27 wrote: post #34

I started doing some experimentation with this new Deep Think agent, and after five prompts I reached my daily usage limit. For $250 USD/mo that’s what you’ll be getting folks. It’s just bizarrely uncompetitive with o3-pro and Grok 4 Heavy. Anecdotally (from my experience) this was the one feature that enthusiasts in the AI community were interested in to justify the exorbitant price of Google’s Ultra subscription. I…

> I find it astonishing that the same company providing free usage of their top models to everybody via AI Studio is nickel-and-diming their actual customers like that. I agree that’s not a good posture, but it is entirely unsurprising. Google is probably not profiting from AI Ultra customers either, and grabbing all that sweet usage data from the free tier of AI Studio is what matters most to improve their models. G…

I'm burning well over 10 millions tokens a day on free tier. 99% of the input is freely availzble data, the rest is useless. I never provided any feedback. Sure there is some telemetry, they can have it.

I doubt I'm an isolated case. This Gemini gig will cost Google a lot, they pushed it on all android phones around the globe. I can't wait to see what happens when they have to admit that not many people will pay over 20 bucks for "Ai", and I would pay well over 20 bucks just to see the face of the c suite next year when one dares to explain in simple terms there is absolutely no way to recoup the DC investment and that powering the whole thing will cost the company 10 times that.

Re: Gemini 2.5 Deep Think

#198

Post by hirako2000 »

LeafItAlone wrote: post #175
epa wrote: post #146

Earlier quoted context omitted.

Unfortunately we are still in the prompt optimization stage, garbage in garbage out

Maybe. That could be true. But doesn’t happen the same with other tools. I’ll give the same exact prompt to all of LLMs I have access to and look at the responses for the best one. Grok is consistently the worst. So if it’s garbage in, garbage out, why are the other ones so much better at dealing with my garbage?

I think it meant in the training stage, not inference.

Re: Gemini 2.5 Deep Think

#199

Post by jiggawatts »

robwwilliams wrote: post #191
nyrikki wrote: post #134

Earlier quoted context omitted.

IIRC that isn't possible under current models at least in general, for multiple reasons, including attention cannot attend to future tokens, the fact that they are existential logic, that they are really NLP and not NLU, etc... Even proof mining and the Harrop formula have to exclude disjunction and existential quantification to stay away from intuitionist math. IID in PAC/ML implies PEM which is also intentionally e…

Hmm, I needed Claude 4’s help to parse your response. The critique was not too kind to your abbreviated arguments that current systems are not able to gauge the complexity of a prompt and the resources needed to address a question.

It feels like the rant of someone upset that their decades of formal logic approach to AI become a dead end.

I see this semi-regularly: futile attempts at handwaving away the obvious intelligence by some formal argument that is either irrelevant or inapplicable. Everything from thermodynamics — which applies to human brains too — to information theory.

Grey-bearded academics clinging to anything that might float to rescue their investment into ineffective approaches.

PS: This argument seems to be that LLMs “can’t think ahead” when all evidence is that they clearly can! I don’t know exactly what words I’ll be typing into this comment textbox seconds or minutes from now but I can — hopefully obviously — think intelligent thoughts and plan ahead.

PPS: The em-dashes were inserted automatically by my iPhone, not a chat bot. I assure you that I am a mostly human person.

Re: Gemini 2.5 Deep Think

#200

Post by hirako2000 »

llm_nerd wrote: post #49
svantana wrote: post #45

Earlier quoted context omitted.

I suspect that the main goal here was to grab the top spot in a bunch of benchmarks, and being counted as an "available" model.

They're using it as a major inducement to upgrade to AI Ultra. I mean, the image and video stuff is neat, but adds no value for the vast majority of AI subscribers, so right now this is the most notable benefit of paying 12x more. FWIW, Google seems to be having some severe issues with oddball, perhaps malfunctioning quota systems. I'm regularly finding extraordinarily little use of gemini-cli is hitting the purporte…

I faced the exact same problem, with the API. It seems that it doesn't throttle early enough, then may cumulate the cool off period, malong it impossible to determine when to fire requests again.

Also, I noticed Gemini (even flash) has Google search support. But only via the web UI or the native mobile app. Via the API that would requires serp via MCP of sort. Even with Gemini pro.

Oh, some models are regularly facing outages. 503s are not uncommon. No SLA page, alerts, whatsoever.

The reasoning feature is buggy, even if disabled, it sometimes triggers anyway.

It occured to me the other day that Google probably have the best engineers given how good Gemini performs and where it's coming from, and the context window that is uniquely large compared to any other model. But that it is likely operated by managers coming from AWS where shipping half baked, barely tested software, was all it took to get a bonus.

Post reply on HN