Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 3 Flash: Frontier intelligence built for speed

blog.google

Post reply on HN
Print view View original

461–470 of 609 posts

Re: Gemini 3 Flash: Frontier intelligence built for speed

#461

Post by danpalmer »

jrk wrote: post #411
danpalmer wrote: post #310

Earlier quoted context omitted.

Hardware is a factor here. GPUs are necessarily higher latency than TPUs for equivalent compute on equivalent data. There are lots of other factors here, but latency specifically favours TPUs. The only non-TPU fast models I'm aware of are things running on Cerebras can be much faster because of their CPUs, and Grok has a super fast mode, but they have a cheat code of ignoring guardrails and making up their own world…

Why are GPUs necessarily higher latency than TPUs? Both require roughly the same arithmetic intensity and use the same memory technology at roughly the same bandwidth.

My understanding is that TPUs do not use memory in the same way. GPUs need to do significantly more store/fetch operations from HBM, where TPUs pipeline data through systolic arrays far more. From what I've heard this generally improves latency and also reduces the overhead of supporting large context windows.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#462

Post by danpalmer »

nl wrote: post #409
danpalmer wrote: post #310

Earlier quoted context omitted.

Hardware is a factor here. GPUs are necessarily higher latency than TPUs for equivalent compute on equivalent data. There are lots of other factors here, but latency specifically favours TPUs. The only non-TPU fast models I'm aware of are things running on Cerebras can be much faster because of their CPUs, and Grok has a super fast mode, but they have a cheat code of ignoring guardrails and making up their own world…

> GPUs are necessarily higher latency than TPUs for equivalent compute on equivalent data. Where are you getting that? All the citations I've seen say the opposite, eg: > Inference Workloads: NVIDIA GPUs typically offer lower latency for real-time inference tasks, particularly when leveraging features like NVIDIA's TensorRT for optimized model deployment. TPUs may introduce higher latency in dynamic or low-batch-size…

I thought it was generally accepted that inference was faster on TPUs. This was one of my takeaways from the LLM scaling book: https://jax-ml.github.io/scaling-book/ – TPUs just do less work, and data needs to move around less for the same amount of processing compared to GPUs. This would lead to lower latency as far as I understand it.

The citation link you provided takes me to a sales form, not an FAQ, so I can't see any further detail there.

> Both Cerebras and Grok have custom AI-processing hardware (not CPUs).

I'm aware of Cerebras' custom hardware. I agree with the other commenter here that I haven't heard of Grok having any. My point about knowledge grounding was simply that Grok may be achieving its latency with guardrail/knowledge/safety trade-offs instead of custom hardware.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#463

Post by jdanbrown »

jaigupta wrote: post #454

Only if I could figure out how to use it. I have been using Claude Code and enjoy it. I sometimes also try Codex which is also not bad. Trying to use Gemini cli is such a pain. I bought GDP Premium and configured GCP, setup environment variables, enabled preview features in cli and did all the dance around it and it won't let me use gemini 3. Why the hell I am even trying so hard?

Have you tried OpenRouter (https://openrouter.ai)? I’ve been happy using it as a unified api provider with great model coverage (including Google, Anthropic, OpenAI, Grok, and the major open models). They charge 5% on top of each model’s api costs, but I think it’s worth it to have one centralized place to insert my money and monitor my usage. I like being able to switch out models without having to change my tools, and I like being able to easily head-to-head compare claude/gemini/gpt when I get stuck on a tricky problem.

Then you just have to find a coding tool that works with OpenRouter. Afaik claude/codex/cursor don’t, at least not without weird hacks, but various of the OSS tools do — cline, roo code, opencode, etc. I recently started using opencode (https://github.com/sst/opencode), which is like an open version of claude code, and I’ve been quite happy with it. It’s a newer project so There Will Be Bugs, but the devs are very active and responsive to issues and PRs.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#464

Post by grog454 »

nl wrote: post #456
grog454 wrote: post #433

Earlier quoted context omitted.

I guess there's two things I'm still stuck on: 1. What is the purpose of the benchmark? 2. What is the purpose of publicly discussing a benchmark's results but keeping the methodology secret? To me it's in the same spirit as claiming to have defeated alpha zero but refusing to share the game.

1. The purpose of the benchmark is to choose what models I use for my own system(s). This is extremely common practice in AI - I think every company I've worked with doing LLM work in the last 2 years has done this in some form. 2. I discussed that up-thread, but https://github.com/microsoft/private-benchmarking and https://arxiv.org/abs/2403.00393 discuss some further motivation for this if you are interested. > To…

I see the potential value of private evaluations. They aren't scientific but you can certainly beat a "vibe test".

I don't understand the value of a public post discussing their results beyond maybe entertainment. We have to trust you implicitly and have no way to validate your claims.

> There is no "winning" at benchmarks, it's simply that it is a better and more repeatable evaluation than the old "vibe test" that people did in 2024.

Then you must not be working in an environment where a better benchmark yields a competitive advantage.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#465

Post by android521 »

jaigupta wrote: post #454

Only if I could figure out how to use it. I have been using Claude Code and enjoy it. I sometimes also try Codex which is also not bad. Trying to use Gemini cli is such a pain. I bought GDP Premium and configured GCP, setup environment variables, enabled preview features in cli and did all the dance around it and it won't let me use gemini 3. Why the hell I am even trying so hard?

use cursor. it allows you to choose any model to use.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#466

Post by cmckn »

willis936 wrote: post #405
exegete wrote: post #272

Earlier quoted context omitted.

Apple Intelligence is going to be Gemini https://www.macrumors.com/2025/11/05/apple-siri-google-gemin...

That's too bad. Apple's most interesting value proposition is running local inference with big privacy promises. They wouldn't need to be the highest performer to offer something a lot of people might want.

My understanding is Apple will be hosting Gemini models themselves on the private compute system they announced a while back.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#467

Post by strangegecko »

eru wrote: post #422
mips_avatar wrote: post #368

Earlier quoted context omitted.

It's weird they don't document this stuff. Like understanding things like tool call latency and time to first token is extremely important in application development.

Humans often answer with fluff like "That's a good question, thanks for asking that, [fluff, fluff, fluff]" to give themselves more breathing room until the first 'token' of their real answer. I wonder if any LLM are doing stuff like that for latency hiding?

Do humans really do that often?

Coming up with all that fluff would keep my brain busy, meaning there's actually no additional breathing room for thinking about an answer.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#468

Post by grog454 »

grog454 wrote: post #440
akoboldfrying wrote: post #432

Earlier quoted context omitted.

I actually think "concealing the question" is not only a good idea, but a rather general and powerful idea that should be much more widely deployed (but often won't be, for what I consider "emotional reasons"). Example: You are probably already aware that almost any metric that you try to use to measure code quality can be easily gamed. One possible strategy is to choose a weighted mixture of metrics and conceal the…

It's hard to have any certainty around concealment unless you are only testing local LLMs. As a matter of principle I assume the input and output of any query I run in a remote LLM is permanently public information (same with search queries). Will someone (or some system) see my query and think "we ought to improve this"? I have no idea since I don't work on these systems. In some instances involving random sampling.…

I learned in another thread there is some work being done to avoid contamination of training data during evaluation of remote models using trusted execution environments (https://arxiv.org/pdf/2403.00393). It requires participation of the model owner.

Re: Gemini 3 Flash: Frontier intelligence built for speed

#469

Post by CamperBob2 »

procaryote wrote: post #307
Workaccount2 wrote: post #299

Earlier quoted context omitted.

I desperately want to be able to real-time dictate actions to take on my phone. Stuff like: "Open Chrome, new tab, search for xyz, scroll down, third result, copy the second paragraph, open whatsapp, hit back button, open group chat with friends, paste what we copied and send, send a follow-up laughing tears emoji, go back to chrome and close out that tab" All while being able to just quickly glance at my phone. Ther…

is that faster to say than do, or is it an accessibility or while-driving need?

I don't understand that use case at all. How can you tell it to do all that stuff, if you aren't sitting there glued to the screen yourself?

Re: Gemini 3 Flash: Frontier intelligence built for speed

#470

Post by jaigupta »

jdanbrown wrote: post #463
jaigupta wrote: post #454

Only if I could figure out how to use it. I have been using Claude Code and enjoy it. I sometimes also try Codex which is also not bad. Trying to use Gemini cli is such a pain. I bought GDP Premium and configured GCP, setup environment variables, enabled preview features in cli and did all the dance around it and it won't let me use gemini 3. Why the hell I am even trying so hard?

Have you tried OpenRouter ( https://openrouter.ai )? I’ve been happy using it as a unified api provider with great model coverage (including Google, Anthropic, OpenAI, Grok, and the major open models). They charge 5% on top of each model’s api costs, but I think it’s worth it to have one centralized place to insert my money and monitor my usage. I like being able to switch out models without having to change my tools…

I have used OpenRouter before but in this case I was trying to use it like Claude Code (agentic coding with a simple fixed monthly subscription). I don't want to pay per use via direct APIs as I am afraid it might have surprising bills. My point was, why Google makes it so damn hard even for paid subscriptions where it was supposed to work.
Post reply on HN