Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ironwood: The first Google TPU for the age of inference

blog.google

Post reply on HN
Print view View original

111–120 of 186 posts

Re: Ironwood: The first Google TPU for the age of inference

#111

Post by DisjointedHunt »

Cloud resources are trending towards consumer technology adoption numbers rather than being reserved mostly for Enterprise. This is the most exciting thing in decades!

There is going to be a GPU/Accelerator shortage for the foreseeable future to run the most advanced models, Gemini 2.5 Pro is such a good example. It is probably the first model that many developers i've considered skeptics of extended agent use have started to saturate free token thresholds on.

Grok is honestly the same, but the lack of an API is suggestive of the massive demand wall they face.

Re: Ironwood: The first Google TPU for the age of inference

#113

Post by shitpostbot »

theptip wrote: post #103
nharada wrote: post #12

The first specifically designed for inference? Wasn’t the original TPU inference only?

The phrasing is very precise here, it’s the first TPU for _the age of inference_, which is a novel marketing term they have defined to refer to CoT and Deep Research.

They didn't though?

> first designed specifically for inference. For more than a decade, TPUs have powered Google’s most demanding AI training and serving workloads...

What do they think serving is? I think this marketing copy was written by someone with no idea what they are talking about, and not reviewed by anyone who did.

Also funny enough it kinda looks like they've scrubbed all their references to v4i, where the i stands for inference. https://gwern.net/doc/ai/scaling/hardware/2021-jouppi.pdf

Re: Ironwood: The first Google TPU for the age of inference

#114

Post by fancyfredbot »

dylan604 wrote: post #74
scarmig wrote: post #71

Earlier quoted context omitted.

No one does anything useful with a $3 spend. That's not anything particular to TPUs, though.

That's my point. The touting of $3 is beyond misleading.

You can do real work for a few hundred dollars which is hardly the exclusive domain of "whales"?

The programmer who writes code to run on these likely costs at least 15x this amount an hour.

Re: Ironwood: The first Google TPU for the age of inference

#115

Post by mmx1 »

huijzer wrote: post #105
miki123211 wrote: post #100

Earlier quoted context omitted.

Wow, you guys needed a custom ASIC for inference before CNNs were even invented ? What were the use cases like back then?

According to a Google blog post from 2016 [1], use-cases were RankBrain to improve the relevancy of search results and Street View. Also they used it for AlphaGo. And from what I remember from my MSc thesis, they also probably were starting to use it for Translate. I can't find any TPU reference in the Attention is All You Need or BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, but I…

Yes, IIRC (please correct me if I'm wrong), translate did utilize Seastar (TPU v1) which was integer only, so not easily useful for training.

Re: Ironwood: The first Google TPU for the age of inference

#116

Post by sodality2 »

01HNNWZ0MV43FF wrote: post #93
crazygringo wrote: post #60

Earlier quoted context omitted.

You can't get excited about lower prices for your cloud GPU workloads thanks to the competition it brings to Nvidia? This benefits everyone, even if you don't use Google Cloud, because of the competition it introduces.

I like owning things

Cloud will buy less NVDA chips, and since they're related goods, prices will drop.

Re: Ironwood: The first Google TPU for the age of inference

#117

Post by xadhominemx »

01HNNWZ0MV43FF wrote: post #93
crazygringo wrote: post #60

Earlier quoted context omitted.

You can't get excited about lower prices for your cloud GPU workloads thanks to the competition it brings to Nvidia? This benefits everyone, even if you don't use Google Cloud, because of the competition it introduces.

I like owning things

You own any GB200s?

Re: Ironwood: The first Google TPU for the age of inference

#118

Post by xadhominemx »

justanotheratom wrote: post #76
throwaway48476 wrote: post #26

Its hard to be excited about hardware that will only exist in the cloud before shredding.

exactly. I wish Groq would start selling their cards that they use internally.

They would lose money on every sale

Re: Ironwood: The first Google TPU for the age of inference

#119

Post by baobabKoodaa »

01HNNWZ0MV43FF wrote: post #93
crazygringo wrote: post #60

Earlier quoted context omitted.

You can't get excited about lower prices for your cloud GPU workloads thanks to the competition it brings to Nvidia? This benefits everyone, even if you don't use Google Cloud, because of the competition it introduces.

I like owning things

You will own nothing and you will be happy.

Re: Ironwood: The first Google TPU for the age of inference

#120

Post by mmx1 »

maxrmk wrote: post #109
throwaway48476 wrote: post #98

Earlier quoted context omitted.

[flagged]

I love to hate on google, but I suspect this is strategic enough that they wont kill it. Like graviton at AWS its as much of a negotiation tool as it is a technical solution, letting them push harder with NVIDIA on pricing because they have a backup option.

Google has done stuff primarily for negotiation purposes (e.g. POWER9 chips) but TPU ain't one. It's not a backup option or presumed "inferior solution" to NVIDIA. Their entire ecosystem is TPU-first.
Post reply on HN