Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our eighth generation TPUs: two chips for the agentic era

blog.google

Post reply on HN
Print view View original

81–90 of 240 posts

Re: Our eighth generation TPUs: two chips for the agentic era

#81

Post by akelly »

gordonhart wrote: post #41
kamranjon wrote: post #39

It's interesting that, of the large inference providers, Google has one of the most inconvenient policies around model deprecation. They deprecate models exactly 1 year after releasing them and force you to move onto their next generation of models. I had assumed, because they are using their own silicon, that they would actually be able to offer better stability, but the opposite seems to be true. Their rate limitin…

It's frustrating how cavalier they are about killing old Gemini releases. My read is that once a new model is serving >90% of volume, which happens pretty quickly as most tools will just run the latest+greatest model, the standard Google cost/benefit analysis is applied and the old thing is unceremoniously switched off. It's actually surprising that they recently extended the EOL date for Gemini 2.5. Google has never…

I've heard GenAI.mil still has Gemini 2.5 only.

Re: Our eighth generation TPUs: two chips for the agentic era

#82

Post by dekhn »

VectorLock wrote: post #75
akersten wrote: post #43

Earlier quoted context omitted.

I'd go long Google too if using Gemini CLI felt anything close to the experience I get with Codex or Claude. They might have great hardware but it's worthless if their flagship coding agent gets stuck in loops trying to find the end of turn token.

I use Claude Code all day and use Gemini CLI for personal projects and I don't see the huge gap that other people seem to talk about a lot. Truthfully there are parts of Gemini CLI I like better than Claude Code.

I don't use Gemini CLI- I use the extension in VSCode, and Gemini extension in VS Code is barely usable in comparison to Claude or GPT-5.4. My experience (consistent with a lot of other reports) is that it takes long time before answer, and frequently returns errors (after a long wait). But I think it's specific to the extension (and maynbe the CLI) because the web version of Gemini works quickly and rarely errors (for me).

Re: Our eighth generation TPUs: two chips for the agentic era

#83

Post by gordonhart »

surajrmal wrote: post #67
gordonhart wrote: post #41

Earlier quoted context omitted.

It's frustrating how cavalier they are about killing old Gemini releases. My read is that once a new model is serving >90% of volume, which happens pretty quickly as most tools will just run the latest+greatest model, the standard Google cost/benefit analysis is applied and the old thing is unceremoniously switched off. It's actually surprising that they recently extended the EOL date for Gemini 2.5. Google has never…

What benefit is there to sticking on older models? If the API is the same, what are the switching costs?

If you're trying to run repeatable workflows, stability from not changing the model can outweigh the benefits of a smarter new model.

The cost can also change dramatically: on top of the higher token costs for Gemini Pro ($1.25/mtok input for 2.5 versus $2/mtok input for 3.1), the newer release also tokenizes images and PDF pages less efficiently by default (>2x token usage per image/page) so you end up paying much much more per request on the newer model.

These are somewhat niche concerns that don't apply to most chat or agentic coding use cases, but they're very real and account for some portion of the traffic that still flows to older Gemini releases.

Re: Our eighth generation TPUs: two chips for the agentic era

#84

Post by hyperbovine »

fourside wrote: post #46
akersten wrote: post #43

Earlier quoted context omitted.

I'd go long Google too if using Gemini CLI felt anything close to the experience I get with Codex or Claude. They might have great hardware but it's worthless if their flagship coding agent gets stuck in loops trying to find the end of turn token.

Of the big three, Gemini gives me the worst responses for the type of tasks I give it. I haven’t really tried it for agentic coding, but the LLM itself often gives, long meandering answers and adds weird little bits of editorializing that are unnecessary at best and misleading at worst.

Same. The tone is really off. Here is a response I just got from Gemini 3.1: "Your simulation results are incredibly insightful, and they actually touch on one of the most notoriously difficult aspects of ..." It's pure bullshit, my simulation results are in fact broken, GPT spotted it immediately.

Re: Our eighth generation TPUs: two chips for the agentic era

#85

Post by gordonhart »

akelly wrote: post #81
gordonhart wrote: post #41

Earlier quoted context omitted.

It's frustrating how cavalier they are about killing old Gemini releases. My read is that once a new model is serving >90% of volume, which happens pretty quickly as most tools will just run the latest+greatest model, the standard Google cost/benefit analysis is applied and the old thing is unceremoniously switched off. It's actually surprising that they recently extended the EOL date for Gemini 2.5. Google has never…

I've heard GenAI.mil still has Gemini 2.5 only.

Wouldn't surprise me. The best model you can get on AWS GovCloud is still Claude Sonnet 4.5.

Re: Our eighth generation TPUs: two chips for the agentic era

#86

Post by akelly »

selectodude wrote: post #45
zozbot234 wrote: post #36

Earlier quoted context omitted.

$15/GB is retail price for DIMM sticks. Is HBM4 really that cheap?

HBM is just DRAM stacked directly next to the die. The expensive part is gluing it on there. The chips themselves are pretty much the same.

HBM uses about twice as much DRAM silicon per GB due to all the space for interconnect

Re: Our eighth generation TPUs: two chips for the agentic era

#87

Post by mrlongroots »

fulafel wrote: post #21

"TPU 8t and TPU 8i deliver up to two times better performance-per-watt over the previous generation" sounds impressive especially as the previous generation is so recent (2025). Interesting that there's separate inference and training focused hardware. Do companies using NV hardware also use different hardware for each task or is their compute more fungible?

That training is compute-bound and inference is memory-bound is well-known, but I don't think Nvidia deployments typically specialize for one vs the other.

One reason is that most clouds/neoclouds don't own workloads, and want fungibility. Given that you're spending a lot on H200s and what not it's good to also spend on the networking to make sure you can sell them to all kinds of customers. The Grok LPU in Vera Rubin is an inference-specific accelerator, and Cerebras is also inference-optimized so specialization is starting to happen.

Re: Our eighth generation TPUs: two chips for the agentic era

#88

Post by kamranjon »

surajrmal wrote: post #67
gordonhart wrote: post #41

Earlier quoted context omitted.

It's frustrating how cavalier they are about killing old Gemini releases. My read is that once a new model is serving >90% of volume, which happens pretty quickly as most tools will just run the latest+greatest model, the standard Google cost/benefit analysis is applied and the old thing is unceremoniously switched off. It's actually surprising that they recently extended the EOL date for Gemini 2.5. Google has never…

What benefit is there to sticking on older models? If the API is the same, what are the switching costs?

Consistency, new models don't behave the same on every task as their predecessors. So you end up building pipelines that rely on specific behavior, but now you find that the new model performs worse with regards to a specific task you were performing, or just behaves differently and needs prompt adjustments. They also can fundamentally change the default model settings during new releases, for example Gemini 2.5 models had completely different behavior with regards to temperature settings than previous models. It just creates a moving target that you constantly have to adjust and rework instead of providing a platform that you and by extension your users can rely on. Other providers have much longer deprecation windows, so they must at least understand this frustration.

Re: Our eighth generation TPUs: two chips for the agentic era

#89

Post by vondur »

pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

Isn't Amazon doing the same thing, making their own TPU's?

Re: Our eighth generation TPUs: two chips for the agentic era

#90

Post by acedTrex »

sigmoid10 wrote: post #13
pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

I'd bet that too if their management wasn't so incredibly uninspiring. Like, Apple under Cook was also pretty mild and a huge step down from Jobs, but Google feels like it fell off a cliff. If it wasn't for OpenAI releasing ChatGPT, they might still be sitting on that tech while only testing it internally. Now it drives their entire chip R&D.

They had no reason to destroy their golden goose, why release something that could hurt their money printing business.

Honestly im rather impressed with how they handled it, they had enough of the infra and org in place to jump at it once the cat was out of the bag.

Sundar declared a code red or whatever and they made it happen. But that could ONLY happen if they had the bedrock of that ability already built.

No one really remembers now that google was a year behind.

Post reply on HN