Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our eighth generation TPUs: two chips for the agentic era

blog.google

Post reply on HN
Print view View original

171–180 of 240 posts

Re: Our eighth generation TPUs: two chips for the agentic era

#171

Post by otterley »

electroly wrote: post #79
fulafel wrote: post #21

"TPU 8t and TPU 8i deliver up to two times better performance-per-watt over the previous generation" sounds impressive especially as the previous generation is so recent (2025). Interesting that there's separate inference and training focused hardware. Do companies using NV hardware also use different hardware for each task or is their compute more fungible?

I can't answer for NVIDIA but AWS has its own training and inference chips, and word on the street is the inference chips are too weak, so some companies are running inference on the training chips.

They stopped producing Inferentia altogether and are only investing in Trainium now. They also announced a partnership with Cerebras not long ago. That should give you a clue.

https://www.cerebras.ai/press-release/awscollaboration

Re: Our eighth generation TPUs: two chips for the agentic era

#172

Post by solarkraft »

amazingamazing wrote: post #5

If ai ends up having a winner I struggle to see how it doesn’t end with Google winning because they own the entire stack, or Apple because they will have deployed the most potentially AI capable edge sites.

I see a significant chance that they’ll continue to blunder the product side. It might still not matter because of their massive distribution, but leaves them open to disruption by a better product (think IE vs. Chrome).

Re: Our eighth generation TPUs: two chips for the agentic era

#173

Post by otterley »

vibe42 wrote: post #29
NitpickLawyer wrote: post #20

Earlier quoted context omitted.

> If the whole AI bubble spectularly collapes Every other news for the past month has been about lacking capacity. Everyone is having scaling issues with more demand than they can cover. Anthropic has been struggling for a few months, especially visible when EU tz is still up and US east coast comes online. Everything grinds to a halt. MS has been pausing new subscriptions for gh Copilot, also because a lack of capac…

Both are possible; increasing demand and bubble collapse. The way this could happen is if model commoditization increases - e.g. some AI labs keep publishing large open models that increasingly close the gap to the closed frontier models. Also, if consumer hardware keep getting better and models get so good that most people can get most of their usage satisfied by smaller models running on their laptop, they won't pa…

I’m going to stick my neck out a bit and predict that model commoditization will never happen as long as humans keep producing new content and innovation for models to train on. Sure, some open models will be good enough to write software against, but that’s but a fraction of the overall market for this technology.

Re: Our eighth generation TPUs: two chips for the agentic era

#174

Post by SXX »

cmrdporcupine wrote: post #153
UncleOxidant wrote: post #107

Earlier quoted context omitted.

IIRC when Gemini 3 Pro came out it was considered to be just about on par with whatever version of Claude was out then (4?). Now Gemini 3 is looking long in the tooth. Considering how many Chinese models have been released since then, and at least 2 or 3 versions of Claude, it's starting to look like Google is kind of sitting still here. Maybe you're right and they'll surprise us soon with a large step improvement ov…

As other people are saying here: the Gemini models are mostly terrible at tool use and long context management. And maybe not quite as good with finicky "detail" parts of coding generally. Where they excel is just total holistic _knowledge_ about the world. I don't like "talking" to it, because I kind of hate its tone, but I find Gemini generally extremely useful for research and analysis tasks and looking up informa…

People who say Gemini is bad at long contexts are so wrong.

You can put whole 50,000 - 70,000 LOC codebase into Gemini 3.1 Pro context making it 800,000+ tokens, give it detailed task and ask for whole changed files back and it will execute it sometimes in one shot, sometimes in two. E.g depend on whatever stack you work with let you see all the errors at once so it can fix everything on single reply.

Yes it will give you back 5-15 files up to 4000 LOC total with only relevant parts changed.

This is terrible inefficient way to burn $10 of tokens in 20 minutes, but attention and 1:1 context retention is truly amazing.

PS: At the same time it is bad at tool use, but this have nothing to do with context.

Re: Our eighth generation TPUs: two chips for the agentic era

#175

Post by awongh »

orbital-decay wrote: post #162
deaux wrote: post #151

Earlier quoted context omitted.

> If I had to guess the pro and flash variants are 5x to 10x smaller than opus and gpt-5 class models. I really doubt it, especially Pro. If anything I wouldn't be surprised if their hardware lets them run bigger models more cheaply and quickly than the others. Pro is probably smaller than GPT 5.4 and Opus 4.6 (looks like 4.7 decreased in size), but 5x seems way too much. IMO Gemini 3 Pro is the most "intelligent" in…

Regarding Anthropic, they used to make best multilingual and generalist models, it's their policy thing, not a capability issue. Claude 3 was best at this, including dead and low-resource languages. Neither modern Claude nor Gemini are remotely close to what Claude 3 was capable of (e.g. zero-shot writing styles). Anthropic basically reversed their "character training" policy and started optimizing their models for c…

The benchmarks don’t seem to say that language ability has gotten worse?

Re: Our eighth generation TPUs: two chips for the agentic era

#176

Post by byproxy »

otterley wrote: post #173
vibe42 wrote: post #29

Earlier quoted context omitted.

Both are possible; increasing demand and bubble collapse. The way this could happen is if model commoditization increases - e.g. some AI labs keep publishing large open models that increasingly close the gap to the closed frontier models. Also, if consumer hardware keep getting better and models get so good that most people can get most of their usage satisfied by smaller models running on their laptop, they won't pa…

I’m going to stick my neck out a bit and predict that model commoditization will never happen as long as humans keep producing new content and innovation for models to train on. Sure, some open models will be good enough to write software against, but that’s but a fraction of the overall market for this technology.

> as long as humans keep producing new content and innovation

Well.. we won't have to as we'll have models to do it for us!

Re: Our eighth generation TPUs: two chips for the agentic era

#177

Post by otterley »

byproxy wrote: post #176
otterley wrote: post #173

Earlier quoted context omitted.

I’m going to stick my neck out a bit and predict that model commoditization will never happen as long as humans keep producing new content and innovation for models to train on. Sure, some open models will be good enough to write software against, but that’s but a fraction of the overall market for this technology.

> as long as humans keep producing new content and innovation Well.. we won't have to as we'll have models to do it for us!

I'm sure you were kidding, but seriously, the fact that AI-produced music pretty much all sounds the same is a good indicator that AI isn't particularly creative.

Re: Our eighth generation TPUs: two chips for the agentic era

#178

Post by eklitzke »

geremiiah wrote: post #104

TPUs are systolic arrays right? So does that mean that Google is using a hetreogenous cluster compromising both GPUs and TPUs, for workloads that don't map well or at all on TPUs?

I can't speak to what every team at Google does, but there are machines with Nvidia GPUs in Borg. However Google charges orgs internally for cpu/memory/gpu/tpu usage and TPUs are *way* more efficient in terms of FLOPS/$ than Nvidia GPUs, so there is a *huge* incentive for teams to use TPUs if they can, especially for teams operating large products.

Re: Our eighth generation TPUs: two chips for the agentic era

#180

Post by ahmadyan »

deaux wrote: post #151
himata4113 wrote: post #64

I already felt that gemini 3 proved what is possible if you train a model for efficiency. If I had to guess the pro and flash variants are 5x to 10x smaller than opus and gpt-5 class models. They produce drastically lower amount of tokens to solve a problem, but they haven't seem to have put enough effort into refinining their reasoning and execution as they produce broken toolcalls and generally struggle with 'agent…

> If I had to guess the pro and flash variants are 5x to 10x smaller than opus and gpt-5 class models. I really doubt it, especially Pro. If anything I wouldn't be surprised if their hardware lets them run bigger models more cheaply and quickly than the others. Pro is probably smaller than GPT 5.4 and Opus 4.6 (looks like 4.7 decreased in size), but 5x seems way too much. IMO Gemini 3 Pro is the most "intelligent" in…

generally speaking

ultra ~ mythos ~ gpt-4.5 ~ 4x behemoth

pro ~ opus ~ 2x maverick

flash ~ sonnet ~ scout ~ other 20-30b active Chinese models

Post reply on HN