Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's First Tensor Processing Unit: Architecture

thechipletter.substack.com

Post reply on HN
Print view View original

81–90 of 197 posts

Re: Google's First Tensor Processing Unit: Architecture

#81

Post by halflings »

ec109685 wrote: post #79
vineyardmike wrote: post #78

Earlier quoted context omitted.

The answer is far weirder - they had a chat bot, and no one even discussed it in the context of search replacements. They didn’t want to release it because they just didn’t think it should be a product. Only after OpenAI actually disrupted search did they start releasing Gemini/Bard which takes advantage of search.

They were afraid to release it because of unaligned output and hallucinations. ChatGPT showed that people could still get value out of something that wasn’t perfect. E.g. they had this in their labs: https://www.theguardian.com/technology/2022/jun/12/google-en... from July, 2022z

Agree re:hallucinations/safety issues, that was likely one of the main blockers.

And here's the sad part: they had this back in 2019... see this paper released in Jan 2020: https://blog.research.google/2020/01/towards-conversational-...

Re: Google's First Tensor Processing Unit: Architecture

#82

Post by nemothekid »

halflings wrote: post #80
neilv wrote: post #64

Earlier quoted context omitted.

There's an old joke explanation about Xerox and PARC, about the difficulty of "pitching a 'paperless office' to a photocopier company". In Google's case, an example analogy would be pitching making something like ChatGPT widely available, when that would disrupt revenue from search engine paid placements, and from ads on sites that people wouldn't need to visit. (So maybe someone says, better to phase it in subtly, a…

This (innovator's dilemma / too afraid of disrupting your own ads business model) is the most common explanation folks are giving for this, but seems to be some sort of post-rationalization of why such a large company full of competent researchers/engineers would drop the ball this hard. My read (having seen some of this on the inside), is that it was a mix of being too worried about safety issues (OMG, the chatbot o…

>There are many ways to monetize a chatbot, OpenAI for example is raking billions in subscription fees.

Compared to Google, OpenAI's billions is peanuts, while costing a fortune to generate. GPT-4 doesn't seem profitable (if it was, would they need to throttle it?)

Re: Google's First Tensor Processing Unit: Architecture

#83

Post by a_wild_dandan »

layer8 wrote: post #2

> However, although tensors describe the relationship between arbitrary higher-dimensional arrays, in practice the TPU hardware that we will consider is designed to perform calculations associated with one and two-dimensional arrays. Or, more specifically, vector and matrix operations. I still don’t understand why the term “tensor” is used if it’s only vectors and matrices.

Every tensor is just a stack of vectors wearing a trench coat.

Re: Google's First Tensor Processing Unit: Architecture

#84

Post by klodolph »

readyplayernull wrote: post #56
beachy wrote: post #51

Earlier quoted context omitted.

For historical precedent see Xerox Parc.

IBM, Intel, Apple's Newton.

The story I like to tell for the Newton is that it was launched before the technology was ready yet. Like the Sega Game Gear. Old video phones. All those tablets that launched before the iPad.

They’re good ideas, but they shipped a few years too early, and the technology to make them work well at a good price point wasn’t available until later. Like, the Sega Game Gear had a cool active matrix LCD screen, but it took six AA batteries and the batteries only lasted like four hours.

Re: Google's First Tensor Processing Unit: Architecture

#85

Post by technofiend »

klodolph wrote: post #84
readyplayernull wrote: post #56

Earlier quoted context omitted.

IBM, Intel, Apple's Newton.

The story I like to tell for the Newton is that it was launched before the technology was ready yet. Like the Sega Game Gear. Old video phones. All those tablets that launched before the iPad. They’re good ideas, but they shipped a few years too early, and the technology to make them work well at a good price point wasn’t available until later. Like, the Sega Game Gear had a cool active matrix LCD screen, but it took…

The Palm Pilot V had a dockable cell phone modem, but the connectivity wasn't integrated into the OS. It worked but only as a demonstration. Then Palm released a model with integrated data, but the BlackBerry came out the same year. You can be first and still if someone comes along with a much more compelling product, that's the end of you.

Google has a few years left as a search company, but their enshittification of results has doomed them to replacement by LLMs. They seem to have forgotten Google pushed out their predecessors by having the best search results. Targeted advertisements don't qualify.

Re: Google's First Tensor Processing Unit: Architecture

#86

Post by WhitneyLand »

layer8 wrote: post #2

> However, although tensors describe the relationship between arbitrary higher-dimensional arrays, in practice the TPU hardware that we will consider is designed to perform calculations associated with one and two-dimensional arrays. Or, more specifically, vector and matrix operations. I still don’t understand why the term “tensor” is used if it’s only vectors and matrices.

It says: tensors describe the relationship between high-d arrays

It does not say: tensors “only” describe the relationship between high-d arrays

The term “tensor” is used because it covers all cases: scalars, vectors, matrices, and higher-dimensional arrays.

Tensors are still a generalization of vectors and matrices.

Note the context: In ML and computer science, they are considered a generalization. From a strict pure math standpoint they can be considered different.

As frustrating as it seems one is not really more right and context is the decider. There are lots of definitions across STEM fields that change based on the context or field they’re applied to.

Re: Google's First Tensor Processing Unit: Architecture

#87

Post by ro_sharp »

nemothekid wrote: post #82
halflings wrote: post #80

Earlier quoted context omitted.

This (innovator's dilemma / too afraid of disrupting your own ads business model) is the most common explanation folks are giving for this, but seems to be some sort of post-rationalization of why such a large company full of competent researchers/engineers would drop the ball this hard. My read (having seen some of this on the inside), is that it was a mix of being too worried about safety issues (OMG, the chatbot o…

> There are many ways to monetize a chatbot, OpenAI for example is raking billions in subscription fees. Compared to Google, OpenAI's billions is peanuts, while costing a fortune to generate. GPT-4 doesn't seem profitable (if it was, would they need to throttle it?)

> GPT-4 doesn't seem profitable (if it was, would they need to throttle it?)

Maybe? Hardware supply isn’t perfectly elastic

Re: Google's First Tensor Processing Unit: Architecture

#88

Post by Laremere »

nl wrote: post #29

On the podcast interview now Groq CEO Jonathon Ross did[1] he talked about the creation of the original TPUs (which he built at Google). Apparently originally it was a FPGA he did in his 20% time because he sat near the team who was having inference speed issues. They got it working, then Jeff Dean did the math and the decided to do an ASIC. Now of course Google should spin off the TPU team as a separate company. It'…

The way I see, NVidia only has a few advantages ordered from most important to least:

1. Reserved fab space.

2. Highly integrated software.

3. Hardware architecture that exists today.

4. Customer relationships.

but all of these aspects are weak in one way or another:

For #1, fab space is tight, and NVidia can strangle its consumer GPU market if it means selling more AI chips at a higher price. This advantage is gone if a competitor makes big bets years in advance, or another company that has a lot of fab space (intel?) is willing to change priorities.

2. Life is good when your proprietary software is the industry standard. Whether this actually matters will depend on the use case heavily.

3. A benefit now, but not for long. It's my estimation that the hardware design for TPUs is fundamentally much simpler than for GPUs. No need for raytracing, texture samplers, or rasterization. Mostly just needs lots of matrix multiplication and memory. Others moving into the space will be able to catch up quickly.

4. Useful to stay in the conversation, but in a field hungry for any advantage, the hardware vendor with the highest FLOPS (or equivalent) per dollar is going to win enough customers to saturate their manufacturing ability.

So overall, I give them a few years, and then the competition is going to be real quite fast.

Re: Google's First Tensor Processing Unit: Architecture

#90

Post by WhitneyLand »

layer8 wrote: post #10
sillysaurusx wrote: post #7

Earlier quoted context omitted.

I was confused as hell for a long time when I first got into ML, until I figured out how to think about tensors in a visual way. You're right: fundamentally ML is about vector and matrix operations (1D and 2D). So then why are most ML programs 3D, 4D, and in a transformer sometimes up to 6D (?!) One reasonable guess is that the third dimension is time. Actually not. It turns out that time is pretty rare in ML, and it…

Thanks for the background! I still don’t think it’s appropriate to call a batch of matrices a tensor.

It is appropriate in ML and computer science. It’s not in pure math.

There are many terms in math and science where the definition changes based on the context.

Post reply on HN