Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's First Tensor Processing Unit: Architecture

thechipletter.substack.com

Post reply on HN
Print view View original

141–150 of 197 posts

Re: Google's First Tensor Processing Unit: Architecture

#141

Post by ThinkBeat »

This is probably a dumb question, that just shows my ignorance but I keep hearing on the consumer end of things that the M1-M4 chips are good at some AI.

The most important for me these days would be Photoshop, Resolve etc, and I have seen those run a lot faster on Apple new proprietary chips than on my older machines.

That may not translate well at all to what this chip can do or what a H100s can do.

But does it translate at all?

Of course Apple is not selling their propritary chips either so for it to be practical Apple would have to release some from of an external, server something stuffed with their GPUs and AI chips

Re: Google's First Tensor Processing Unit: Architecture

#142

Post by ionelaipatioaei »

immibis wrote: post #137
Kelteseth wrote: post #126

Earlier quoted context omitted.

If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course. [0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR [1] https://www.twitch.tv/georgehotz

Subscriber only videos, so nobody can confirm that he did that, nor archive whatever valuable information he released. At least not without paying some money in the next 7-14 days before they're deleted.

https://www.youtube.com/@geohotarchive

Re: Google's First Tensor Processing Unit: Architecture

#143

Post by ants_everywhere »

nl wrote: post #102
Laremere wrote: post #88

Earlier quoted context omitted.

The way I see, NVidia only has a few advantages ordered from most important to least: 1. Reserved fab space. 2. Highly integrated software. 3. Hardware architecture that exists today. 4. Customer relationships. but all of these aspects are weak in one way or another: For #1, fab space is tight, and NVidia can strangle its consumer GPU market if it means selling more AI chips at a higher price. This advantage is gone…

Actually their real advantage is the large set of highly optimised CUDA kernels. This is the thing that lets them outperform AMD chips even on inferior hardware. And the fact that anything new gets written for CUDA first. There is OpenAI's Triton language for this too and people are beginning to use it (shout out to Unsloth here!). > Reserved fab space. While this is true, it's worth noting that the inference only Gr…

Honest question: will AI help AMD catch up with optimized CUDA/ROCM kernels of their own?

Re: Google's First Tensor Processing Unit: Architecture

#144

Post by rhdunn »

foota wrote: post #67
tw04 wrote: post #63

Earlier quoted context omitted.

Where can I buy a TPU v5 to install in my server? If the answer is “cloud”: that’s why NVidia is wiping the floor.

How many people are out there buying H100s for their personal use?

Probably not many. However, 4090s would be a different situation. There are plenty of guides on running LLMs, stable diffusion, etc. on local hardware.

The H100s would be for businesses looking to get into this space.

Re: Google's First Tensor Processing Unit: Architecture

#145

Post by knowriju »

logicchains wrote: post #101
Karrot_Kream wrote: post #99

Earlier quoted context omitted.

Google gets much more scrutiny then smaller companies so it's understandable to be worried. Pretty much any small mistake of theirs turns into clickbait on here and the other tech news sites and you get hundreds of comments about how evil Big Tech is. Of course it's their own fault that their PR hews negative so frequently but still it's understandable why they were so shy.

Sydney when initially released was much less censored and the vast majority of responses online were positive, "this is hilarious/cool", not "OMG Sydney should be banned!".

You have clearly not heard about Tay and Galactica.

Re: Google's First Tensor Processing Unit: Architecture

#146

Post by refulgentis »

Kelteseth wrote: post #126
rurban wrote: post #114

Earlier quoted context omitted.

They have Vulcan, which is cross-compatible. And AMD has ROCm. pytorch is standard and pytorch has ROCm support. And the Google TPU v5 also has pytorch support. We do have a couple of H100's, but I'd love to replace them with AMD's

If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course. [0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR [1] https://www.twitch.tv/georgehotz

Geohot doesn't know what he's talking about and I'm kinda ashamed to see this lazy thinking leak onto HN. There was an article a couple weeks back on AMD open sourcing drivers in the Linux kernel tree that you should look into.

Re: Google's First Tensor Processing Unit: Architecture

#147

Post by Symmetry »

Karrot_Kream wrote: post #99
halflings wrote: post #80

Earlier quoted context omitted.

This (innovator's dilemma / too afraid of disrupting your own ads business model) is the most common explanation folks are giving for this, but seems to be some sort of post-rationalization of why such a large company full of competent researchers/engineers would drop the ball this hard. My read (having seen some of this on the inside), is that it was a mix of being too worried about safety issues (OMG, the chatbot o…

Google gets much more scrutiny then smaller companies so it's understandable to be worried. Pretty much any small mistake of theirs turns into clickbait on here and the other tech news sites and you get hundreds of comments about how evil Big Tech is. Of course it's their own fault that their PR hews negative so frequently but still it's understandable why they were so shy.

It's understandable that people at Google are worried because it's likely very unpleasant to see critical articles and tweets about something you did. But that isn't really bad for Google's business in any of the ways that losing to someone on AI would be.

Re: Google's First Tensor Processing Unit: Architecture

#148

Post by QuadmasterXLII »

KeplerBoy wrote: post #128
Kelteseth wrote: post #126

Earlier quoted context omitted.

If AMD fixes or open sources their proprietary firmware blob[0]. Geohot streamed all weekend on Twitch, reverse engineering the AMD firmware. It was quite entertaining learning about how that low level hardware firmware works[1] and his rants about AMD of course. [0] https://www.phoronix.com/news/Tinybox-Radeon-Again-UMR [1] https://www.twitch.tv/georgehotz

Geohot is wrangling with unsupported consumer hardware. The datacenter stuff is on a different architecture and driver stack. The number one supercomputer on the top500 list (frontier at ORNL) is based on AMD GPUs and AMD is probably more invested in supporting that.

I mean, that's kinda nvidia's whole shtick: anyone can play around synthesizing cat pictures on their gaming GPU and if they make a breakthrough, the same software will transfer to X million dollar supercomputers.

Re: Google's First Tensor Processing Unit: Architecture

#149

Post by Symmetry »

rhelz wrote: post #50
cowsandmilk wrote: post #45

Earlier quoted context omitted.

You’re seeming to imply the number of instructions available is what distinguishes CISC, but it never has been.

Guys....what are the instructions? The on-chip memory they are talking about is essentially...a big register set. So we have load from main memory into registers, store from registers into main memory, multiply matrices--source and dest are stored in registers.... We have a 20 instruction, load-store cpu....how is this not RISC? At least RISC how we used the term in 1995?

I don't think it makes any sense to talk about all on-chip memory as a register set. In practice most uses of REP MOVS these days don't leave L3$ but because it's an instruction that runs for a highly variable amount of time while transferring data between different locations we consider it very CISCy. And the TPU also has instructions to transfer data over PCIe to and from the TPU's local DDR3 memory as well, which isn't on the chip and I hope you would agree that it's not like a register at that point.

If every instruction was always one 256 element unit maybe you could make the analogy stick. But it's working with 256*N element operations.

Re: Google's First Tensor Processing Unit: Architecture

#150

Post by Symmetry »

WhitneyLand wrote: post #86
layer8 wrote: post #2

> However, although tensors describe the relationship between arbitrary higher-dimensional arrays, in practice the TPU hardware that we will consider is designed to perform calculations associated with one and two-dimensional arrays. Or, more specifically, vector and matrix operations. I still don’t understand why the term “tensor” is used if it’s only vectors and matrices.

It says: tensors describe the relationship between high-d arrays It does not say: tensors “only” describe the relationship between high-d arrays The term “tensor” is used because it covers all cases: scalars, vectors, matrices, and higher-dimensional arrays. Tensors are still a generalization of vectors and matrices. Note the context: In ML and computer science, they are considered a generalization. From a strict pur…

Famously whether free helium is a molecule or not depends on whether you're talking to a physicist or a chemist.

But yeah, people in different countries speak different languages and the same sound, like "no" can mean a negation in English but a possessive in Japanese. And as different fields establish their jargons they often redefine words in different ways. It's just something you have to be aware of.

Post reply on HN