Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our eighth generation TPUs: two chips for the agentic era

blog.google

Post reply on HN
Print view View original

41–50 of 240 posts

Re: Our eighth generation TPUs: two chips for the agentic era

#41

Post by gordonhart »

kamranjon wrote: post #39

It's interesting that, of the large inference providers, Google has one of the most inconvenient policies around model deprecation. They deprecate models exactly 1 year after releasing them and force you to move onto their next generation of models. I had assumed, because they are using their own silicon, that they would actually be able to offer better stability, but the opposite seems to be true. Their rate limitin…

It's frustrating how cavalier they are about killing old Gemini releases. My read is that once a new model is serving >90% of volume, which happens pretty quickly as most tools will just run the latest+greatest model, the standard Google cost/benefit analysis is applied and the old thing is unceremoniously switched off. It's actually surprising that they recently extended the EOL date for Gemini 2.5. Google has never been a particularly customer-obsessed company...

Re: Our eighth generation TPUs: two chips for the agentic era

#42

Post by whattheheckheck »

sigmoid10 wrote: post #13
pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

I'd bet that too if their management wasn't so incredibly uninspiring. Like, Apple under Cook was also pretty mild and a huge step down from Jobs, but Google feels like it fell off a cliff. If it wasn't for OpenAI releasing ChatGPT, they might still be sitting on that tech while only testing it internally. Now it drives their entire chip R&D.

What would an inspiring leader do differently for you?

Re: Our eighth generation TPUs: two chips for the agentic era

#43

Post by akersten »

pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

I'd go long Google too if using Gemini CLI felt anything close to the experience I get with Codex or Claude. They might have great hardware but it's worthless if their flagship coding agent gets stuck in loops trying to find the end of turn token.

Re: Our eighth generation TPUs: two chips for the agentic era

#44

Post by WarmWash »

Whats interesting to note, as someone who uses Gemini, ChatGPT, and Claude, is that Gemini consistently uses drastically fewer tokens than the other two. It seems like gemini is where it is because it has a much smaller thinking budget.

It's hard to reconcile this because Google likely has the most compute and at the lowest cost, so why aren't they gassing the hell out of inference compute like the other two? Maybe all the other services they provide are too heavy? Maybe they are trying to be more training heavy? I don't know, but it's interesting to see.

Re: Our eighth generation TPUs: two chips for the agentic era

#45

Post by selectodude »

zozbot234 wrote: post #36
paulmist wrote: post #14

At $15/GB of HBM4 the 331.8TB of HBM4 per pod is 5 million...

$15/GB is retail price for DIMM sticks. Is HBM4 really that cheap?

HBM is just DRAM stacked directly next to the die. The expensive part is gluing it on there. The chips themselves are pretty much the same.

Re: Our eighth generation TPUs: two chips for the agentic era

#46

Post by fourside »

akersten wrote: post #43
pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

I'd go long Google too if using Gemini CLI felt anything close to the experience I get with Codex or Claude. They might have great hardware but it's worthless if their flagship coding agent gets stuck in loops trying to find the end of turn token.

Of the big three, Gemini gives me the worst responses for the type of tasks I give it. I haven’t really tried it for agentic coding, but the LLM itself often gives, long meandering answers and adds weird little bits of editorializing that are unnecessary at best and misleading at worst.

Re: Our eighth generation TPUs: two chips for the agentic era

#47

Post by ihsw »

akersten wrote: post #43
pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

I'd go long Google too if using Gemini CLI felt anything close to the experience I get with Codex or Claude. They might have great hardware but it's worthless if their flagship coding agent gets stuck in loops trying to find the end of turn token.

[dead]

Re: Our eighth generation TPUs: two chips for the agentic era

#48

Post by someguyiguess »

WarmWash wrote: post #44

Whats interesting to note, as someone who uses Gemini, ChatGPT, and Claude, is that Gemini consistently uses drastically fewer tokens than the other two. It seems like gemini is where it is because it has a much smaller thinking budget. It's hard to reconcile this because Google likely has the most compute and at the lowest cost, so why aren't they gassing the hell out of inference compute like the other two? Maybe a…

They have to have SOME competitive advantage. What reason is there to use Gemini over Claude or ChatGPT? It's not producing nearly the quality of output.

Re: Our eighth generation TPUs: two chips for the agentic era

#49

Post by harrall »

echelon wrote: post #25
youniverse wrote: post #15

Earlier quoted context omitted.

Yeah I think there will be a time in a few years (1-2?) when both Google and Apple will get to eat their cake. They aren't playing the same game of speed running unpolished product releases every month to double their valuation. They have time to think and observe and put out something really polished. At least that's the hope! :)

That's because these mega monopolies have diverse income streams and have grown like cancers to tax every system and economy that touches the internet. Anthropic and OpenAI are having to fight like hell to secure market share. Google just gets to sit back and relax with its browser and android monopolies. Why did our regulators fall asleep at the wheel? Google owns 92% of "URL bar" surface area and turned it into a G…

Google invented the AI architecture that Anthropic and OpenAI based their entire companies on? Based off years of research at Google.

Of course they should have to fight with the inventors of the technology they’re using.

Re: Our eighth generation TPUs: two chips for the agentic era

#50

Post by RationPhantoms »

WarmWash wrote: post #44

Whats interesting to note, as someone who uses Gemini, ChatGPT, and Claude, is that Gemini consistently uses drastically fewer tokens than the other two. It seems like gemini is where it is because it has a much smaller thinking budget. It's hard to reconcile this because Google likely has the most compute and at the lowest cost, so why aren't they gassing the hell out of inference compute like the other two? Maybe a…

They just released their enterprise agentic platform today so my expectation is that might be the gravity well for the Fortune 500's to park their inference on.
Post reply on HN