Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Our eighth generation TPUs: two chips for the agentic era

blog.google

Post reply on HN
Print view View original

181–190 of 240 posts

Re: Our eighth generation TPUs: two chips for the agentic era

#181

Post by delbronski »

rtaylorgarlock wrote: post #120
delbronski wrote: post #118

I’ve been using Gemini with Junie (jetbrains attempt at Claude code). While Junie is nowhere near as good as Claude Code, it is way ahead of the current Google tooling. I get quite good consistent results for pretty cheap with this combo.

Do you think that Junie specifically inside the context of jetbrains IDEs and tooling matches the competitors?

I think so. From my experience Claude/codex tooling really excels at vibe coding the whole thing. You give it a folder and just say: now make it do this. And you don’t really care for the code.

Junie tooling excels when you are more involved. Like, look in these two files, add this specific functionality, in this specific way. Junie is usually a lot faster and to the point. Very simple tooling , it just works for this workflow. But it breaks for the “code the whole thing for me” workflow.

Re: Our eighth generation TPUs: two chips for the agentic era

#182

Post by btian »

someguyiguess wrote: post #61
jmyeet wrote: post #56

In recent discussions about Tim Apple [sic] moving on there was a discussion about whether Apple flopped on AI, which is my opinion. Of course you had the false dichotomy of doing nothing or burning money faster than the US military like OpenAI does. IMHO that happy medium is Google. Not having to pay the NVidia tax will likely be a huge competitive advantage. And nobody builds data centers as cost-effectively as Goo…

Apple has not flopped on AI as you say. They are just focused on privacy and are likely waiting for the time when local models become efficient enough to run on iPhones (which is quickly becoming a reality). Google could probably train models for orders of magnitude less money as you say, but they aren't. They are not capable of creating high quality models like OpenAI and Anthropic are. Their company is just too dis…

> They are just focused on privacy and are likely waiting for the time when local models become efficient enough to run on iPhones

How does that make any sense?

iPhones may be able to run local model inference, but Apple still can't train anything if they don't have any data.

Re: Our eighth generation TPUs: two chips for the agentic era

#183

Post by orbital-decay »

awongh wrote: post #175
orbital-decay wrote: post #162

Earlier quoted context omitted.

Regarding Anthropic, they used to make best multilingual and generalist models, it's their policy thing, not a capability issue. Claude 3 was best at this, including dead and low-resource languages. Neither modern Claude nor Gemini are remotely close to what Claude 3 was capable of (e.g. zero-shot writing styles). Anthropic basically reversed their "character training" policy and started optimizing their models for c…

The benchmarks don’t seem to say that language ability has gotten worse?

That's the thing with benchmarks, without evals and actual hands-on experience they can give you false confidence. Claude now sounds almost clinical, and is unable to speak in different styles as easily. Claude 4+ uses a lot more constructions borrowed from English than Claude 3, especially in Slavic languages where they sound unnatural. And most modern models eventually glitch out in longer texts, spitting a few garbage tokens in a random language (Telugu, Georgian, Ukrainian, totally unrelated), then continuing in the main language like nothing happened. It's rare but it happens. Samplers do not help with this, you need a second run to spellcheck it. This wasn't a problem in older models, it's a widespread issue that roughly correlates with the introduction of reasoning. Another new failure mode is self-correction in complicated texts that need reading comprehension: if the model hallucinates an incorrect fact and spots it, it tries to justify or explain it immediately. Which is much more awkward than leaving it incorrect, and also those hallucinations are more common now (maybe because the model learns to make those mistakes together with the correction? I don't know.)

Re: Our eighth generation TPUs: two chips for the agentic era

#184

Post by awongh »

orbital-decay wrote: post #183
awongh wrote: post #175

Earlier quoted context omitted.

The benchmarks don’t seem to say that language ability has gotten worse?

That's the thing with benchmarks, without evals and actual hands-on experience they can give you false confidence. Claude now sounds almost clinical, and is unable to speak in different styles as easily. Claude 4+ uses a lot more constructions borrowed from English than Claude 3, especially in Slavic languages where they sound unnatural. And most modern models eventually glitch out in longer texts, spitting a few gar…

Not disputing this might be true, but this seems like something that should be capturable in a multi-lingual benchmark.

Maybe it's just something that people aren't bothered with?

Re: Our eighth generation TPUs: two chips for the agentic era

#185

Post by amelius »

pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

Don't build your castle in someone else's kingdom.

Buying from nvidia is the only real option and even that is not optimal.

Re: Our eighth generation TPUs: two chips for the agentic era

#186

Post by Jensson »

big-chungus4 wrote: post #139
himata4113 wrote: post #64

I already felt that gemini 3 proved what is possible if you train a model for efficiency. If I had to guess the pro and flash variants are 5x to 10x smaller than opus and gpt-5 class models. They produce drastically lower amount of tokens to solve a problem, but they haven't seem to have put enough effort into refinining their reasoning and execution as they produce broken toolcalls and generally struggle with 'agent…

Am I tripping or is this an AI reply? Like it barely has anything to do with the article other than both are related to AI

An AI reply would be more relevant to the headline / article, humans often write something tangential since we have more going on in our head and not just the context at hand while AI can't ignore context.

Re: Our eighth generation TPUs: two chips for the agentic era

#187

Post by orbital-decay »

awongh wrote: post #184
orbital-decay wrote: post #183

Earlier quoted context omitted.

That's the thing with benchmarks, without evals and actual hands-on experience they can give you false confidence. Claude now sounds almost clinical, and is unable to speak in different styles as easily. Claude 4+ uses a lot more constructions borrowed from English than Claude 3, especially in Slavic languages where they sound unnatural. And most modern models eventually glitch out in longer texts, spitting a few gar…

Not disputing this might be true, but this seems like something that should be capturable in a multi-lingual benchmark. Maybe it's just something that people aren't bothered with?

Basically everyone who experiments with creative writing is keenly aware of that (e.g. roleplayers), it's just the devs that have the experience training the models for it (Anthropic, DeepMind) aren't bothered doing this anymore since there's no money in it.

>this seems like something that should be capturable in a multi-lingual benchmark

Creative writing benchmarks just don't have good objectives to measure against. In particular, valid but inauthentic language constructions can't be captured well if your LLM judge lacks fidelity to capture it to begin with. Which is I think what typically happens.

An easy litmus test would be making a selected character in a story speak Ebonics or Haitian Creole or TikTok. Claude 3 Opus was light years ahead of any model in authenticity in using them, and it was immediately obvious in a side-by-side comparison with any model including Claude 3.5+. Nuances of Polish or Russian profanities/mat or British obscenities are always the hardest for any model (they tend to either swear like dockers or tone it down, lacking the eloquence), but Opus 3 was also ahead in any of those.

Re: Our eighth generation TPUs: two chips for the agentic era

#189

Post by manquer »

smlacy wrote: post #179

> TPU 8i pairs 288 GB of high-bandwidth memory with 384 MB of on-chip SRAM Wow. Just Wow. I presume that's for each chip, and there are 1152 chips in a pod so that's 331TB HBM and 442TB SRAM per pod. Just wow.

That would be 442GB not TB of SRAM.

Aggregating on-chip SRAM is not a useful metric. AFAIK there are no interconnect between two chips at SRAM level so they cannot be shared .

Re: Our eighth generation TPUs: two chips for the agentic era

#190

Post by bossyTeacher »

akersten wrote: post #43
pmb wrote: post #11

At this point, when you are doing big AI you basically have to buy it from NVidia or rent it from Google. And Google can design their chips and engine and systems in a whole-datacenter context, centralizing some aspects that are impossible for chip vendors to centralize, so I suspect that when things get really big, Google's systems will always be more cost-efficient. (disclosure: I am long GOOG, for this and a few o…

I'd go long Google too if using Gemini CLI felt anything close to the experience I get with Codex or Claude. They might have great hardware but it's worthless if their flagship coding agent gets stuck in loops trying to find the end of turn token.

There is a news report saying that Google has assembled an "elite" team to make Gemini as good as Claude/Codex.
Post reply on HN