Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's First Tensor Processing Unit: Architecture

thechipletter.substack.com

Post reply on HN
Print view View original

121–130 of 197 posts

Re: Google's First Tensor Processing Unit: Architecture

#121

Post by Culonavirus »

otabdeveloper4 wrote: post #112
Laremere wrote: post #88

Earlier quoted context omitted.

The way I see, NVidia only has a few advantages ordered from most important to least: 1. Reserved fab space. 2. Highly integrated software. 3. Hardware architecture that exists today. 4. Customer relationships. but all of these aspects are weak in one way or another: For #1, fab space is tight, and NVidia can strangle its consumer GPU market if it means selling more AI chips at a higher price. This advantage is gone…

CUDA is absolute shit, segfaults or compiler errors if you look at it wrong. NVidia's software is the only reason I'm not using GPU's for ML tasks and likely never will.

Skill issue.

Re: Google's First Tensor Processing Unit: Architecture

#122

Post by pyb »

nl wrote: post #29

On the podcast interview now Groq CEO Jonathon Ross did[1] he talked about the creation of the original TPUs (which he built at Google). Apparently originally it was a FPGA he did in his 20% time because he sat near the team who was having inference speed issues. They got it working, then Jeff Dean did the math and the decided to do an ASIC. Now of course Google should spin off the TPU team as a separate company. It'…

There seem to be conflicting reports as to who came up with the TPU https://mastodon.social/@danluu/109641269333636407

Re: Google's First Tensor Processing Unit: Architecture

#123

Post by Culonavirus »

tw04 wrote: post #60
hipadev23 wrote: post #49

How is it that Google invented the TPU and Google Research came up with the paper on LLM and NVDA and AI startup companies have captured ~100% of the value

Because Google can’t focus on a product for more than 18 months if it isn’t generating several billion in PROFIT. They are punch drunk on advertising.

They're like a hyperactive dog chasing its own tail. How many projects did they create only to shut them a bit later? All because there's always some nonsense to chase. Meanwhile the AI train has left the station without them and their search is now an ad infested hot piece of garbage. Don't even get me started on their customer/dev support or how aging things like Google Translate api got absolutely KILLED by GPT-4 like apis overnight.

Google has stage 4 leadership incompetency and can't be helped. The only humane option is euthanasia.

Re: Google's First Tensor Processing Unit: Architecture

#124

Post by Rinzler89 »

klodolph wrote: post #84
readyplayernull wrote: post #56

Earlier quoted context omitted.

IBM, Intel, Apple's Newton.

The story I like to tell for the Newton is that it was launched before the technology was ready yet. Like the Sega Game Gear. Old video phones. All those tablets that launched before the iPad. They’re good ideas, but they shipped a few years too early, and the technology to make them work well at a good price point wasn’t available until later. Like, the Sega Game Gear had a cool active matrix LCD screen, but it took…

>and the batteries only lasted like four hours

Still more than the OG Steam Deck today :)

Re: Google's First Tensor Processing Unit: Architecture

#125

Post by Kelteseth »

KeplerBoy wrote: post #118
fnbr wrote: post #42

Earlier quoted context omitted.

I’m surprised they sell any to external customers, to be honest.

They don't sell any TPUs, do they? Besides the, now ancient, coral toy-TPUs.

Has there been any development? The last update is from 2021 [0], but it is not officially killed by google(.com)

[0] https://coral.ai/news/updates-07-2021

Re: Google's First Tensor Processing Unit: Architecture

#126

Post by Kelteseth »

rurban wrote: post #114
nl wrote: post #103

Earlier quoted context omitted.

Yes, but they don't have the custom kernels that CUDA has. TPUs do have some!

They have Vulcan, which is cross-compatible. And AMD has ROCm. pytorch is standard and pytorch has ROCm support. And the Google TPU v5 also has pytorch support. We do have a couple of H100's, but I'd love to replace them with AMD's

If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course.

[0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR

[1] https://www.twitch.tv/georgehotz

Re: Google's First Tensor Processing Unit: Architecture

#127

Post by willvarfar »

earthnail wrote: post #115
wstrange wrote: post #59

Earlier quoted context omitted.

It's far too early to suggest Google will not capture value from AI. They have plenty of opportunity to integrate AI into their products.

Microsoft is sooooo far ahead in this game, it’s borderline ridiculous. Google really missed an opportunity to grab major market share from MS Office.

Yes, this! Google Docs is basically basic. But imagine if, years ago, Google had added built-in LLM-based auto-complete and refactoring and summation tools to documents and presentations etc, years ago...

Oh to dream.

Re: Google's First Tensor Processing Unit: Architecture

#128

Post by KeplerBoy »

Kelteseth wrote: post #126
rurban wrote: post #114

Earlier quoted context omitted.

They have Vulcan, which is cross-compatible. And AMD has ROCm. pytorch is standard and pytorch has ROCm support. And the Google TPU v5 also has pytorch support. We do have a couple of H100's, but I'd love to replace them with AMD's

If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course. [0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR [1] https://www.twitch.tv/georgehotz

Geohot is wrangling with unsupported consumer hardware.

The datacenter stuff is on a different architecture and driver stack. The number one supercomputer on the top500 list (frontier at ORNL) is based on AMD GPUs and AMD is probably more invested in supporting that.

Re: Google's First Tensor Processing Unit: Architecture

#129

Post by otabdeveloper4 »

Culonavirus wrote: post #121
otabdeveloper4 wrote: post #112

Earlier quoted context omitted.

CUDA is absolute shit, segfaults or compiler errors if you look at it wrong. NVidia's software is the only reason I'm not using GPU's for ML tasks and likely never will.

Skill issue.

No, CUDA's botched gcc implementation segfaulting due to compiler errors during compilation is not a "skill issue".

(Well, a skill issue of whoever is patching gcc on Nvidia's end, I guess.)

Re: Google's First Tensor Processing Unit: Architecture

#130

Post by otabdeveloper4 »

KeplerBoy wrote: post #116
otabdeveloper4 wrote: post #112

Earlier quoted context omitted.

CUDA is absolute shit, segfaults or compiler errors if you look at it wrong. NVidia's software is the only reason I'm not using GPU's for ML tasks and likely never will.

That's just C. If you're accessing your arrays out of bounds it's going to segfault. hopefully. Can't blame CUDA for that one.

I'm talking about the compiler segfaulting, not the end-user code.
Post reply on HN