Live data from Hacker News

TPUs vs. GPUs and why Google is positioned to win AI race in the long term

uncoveralpha.com

101–110 of 328 posts

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#101
post #56

Earlier quoted context omitted.

We used to call these things DSPs

What is the difference between a DSP and Asic? Is a GPU a DSP?

ASICs bake one algorithm into the chip. DSPs are programmable, like GPUs or CPUs. The thing that historically set them apart were MAC/FMA and zero overhead loops. Then there are all the nice to haves, like built in tables of FFT twiddle factors, helpers for 1D convolution, vector instructions, fixed point arithmetic, etc.

What makes a DSP different from a GPU is the algorithms typically do not scale nicely to large matrices and vectors. For example, recursive filters. They are also usually much cheaper and lower power, and the reason they lost popularity was because Arm MCUs got good enough and economy of scale kicked in.

I've written code for DSPs both in college and professionally. It's much like writing code for CPUs or MCUs (it's all C or C++ at the end of the day). But it's very different from writing compute shaders or designing an ASIC.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#102

Earlier quoted context omitted.

Google Hangouts wasn't small. Google+ was big and supposedly "the future" and is the canonical example of a huge misallocation of resources. Google will have no problem discontinuing Google "AI" if they finally notice that people want a computer to shut up rather than talk at them.

> Google+ was big how you define big? My understanding they failed to compete with facebook, and decided to redirect resources somewhere else.

They put a lot of effort into it, but it never had much usage.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#103
post #20
post #2

A question I don't see addressed in all these articles: what prevents Nvidia from doing the same thing and iterating on their more general-purpose GPU towards a more focused TPU-like chip as well, if that turns out to be what the market really wants.

They will, I'm sure. The big difference is that Google is both the chip designer *and* the AI company. So they get both sets of profits. Both Google and Nvidia contract TSMC for chips. Then Nvidia sells them at a huge profit. Then OpenAI (for example) buys them at that inflated rate and them puts them into production. So while Nvidia is "selling shovels", Google is making their own shovels and has their own mines.

So when the bubble pops the companies making the shovels (TSMC, NVIDIA) might still have the money they got for their products and some of the ex-AI companies might least be able to sell standard compliant GPUs on the wider market.

And Google will end up with lots of useless super specialized custom hardware.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#104
All this assumes that LLMs are the sole mechanism for AI and will remain so forever: no novel architectures (neither hardware nor software), no progress in AI theory, nothing better than LLMs, simply brute force LLM computation ad infinitum.

Perhaps the assumptions are true. The mere presence of LLMs seems to have lowered the IQ of the Internet drastically, sopping up financial investors and resources that might otherwise be put to better use.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#105
post #41

Google's real moat isn't the TPU silicon itself—it's not about cooling, individual performance, or hyper-specialization—but rather the massive parallel scale enabled by their OCS interconnects. To quote The Next Platform: "An Ironwood cluster linked with Google’s absolutely unique optical circuit switch interconnect can bring to bear 9,216 Ironwood TPUs with a combined 1.77 PB of HBM memory... This makes a rackscale…

Also, Google owns the entire vertical stack, which is what most people need. It can provide an entire spectrum of AI services far cheaper, at scale (and still profitable) via its cloud. Not every company needs to buy the hardware and build models, etc., etc.; what most companies need is an app store of AI offerings they can leverage. Google can offer this with a healthy profit margin, while others will eventually run…

Google's work on Jax, pytorch, tensorflow, and the more general XLA underneath are exactly the kind of anti-moat everyone has been clamoring for.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#106

I have read in the past that ASICs for LLMs are not as simple a solution compared to cryptocurrency. In order to design and build the ASIC you need to commit to a specific architecture: a hashing algorithm for a cryptocurrency is fixed but the LLMs are always changing. Am I misunderstanding "TPU" in the context of the article?

"Application-specific" doesn't necessarily mean unprogrammable. Bitcoin miners aren't programmable because they don't need to be. TPUs are ASICs for ML and need to be programmable so they can run different models. In theory, you could make an ASIC hardcoded for a specific model, but given how fast models evolve, it probably wouldn't make much economic sense.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#107

Earlier quoted context omitted.

What is the difference between a DSP and Asic? Is a GPU a DSP?

A DSP contains analog to digital and digital to analog converters plus DMA for fast transfers to main memory and fixed function blocks for finite impulse response and infinite pulse response filters. The fact that they also support vector operations or matrix multiplication is kind of irrelevant and not a defining characteristic of DSPs. If you want to go that far, then everything is a DSP, because all signals are an…

I've worked on DSP's with none of those things. Well, they did have DMA.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#108
post #95

Earlier quoted context omitted.

China will invade Taiwan when they start losing, not when they're increasingly winning. As long as "tomorrow" is a better day to invade Taiwan than today is, China will wait for tomorrow.

Their demographics beg to differ.

If demographics were a big deal, it'd be part of the same "better to invade today or tomorrow" calculation.

Zeihan's predictions on China have been fabulously wrong for 20+ years now.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#109

All this assumes that LLMs are the sole mechanism for AI and will remain so forever: no novel architectures (neither hardware nor software), no progress in AI theory, nothing better than LLMs, simply brute force LLM computation ad infinitum . Perhaps the assumptions are true. The mere presence of LLMs seems to have lowered the IQ of the Internet drastically, sopping up financial investors and resources that might oth…

That's incorrect. TPUs can support many ML workloads, they're not exclusive to LLMs.

Re: TPUs vs. GPUs and why Google is positioned to win AI race in the long term

#110
post #66
post #50

This is the “Microsoft will dominate the Internet” stage. The truth is the LLM boom has opened the first major crack in Google as the front page of the web (the biggest since Facebook), in the same way the web in the long run made Windows so irrelevant Microsoft seemingly don’t care about it at all.

Exactly, ChatGPT pretty much ate away ad volume & retention if th already garbage search results weren't enough. Don't even get me started on Android & Android TV as an ecosystem.

That's not the story that GOOGs quarterly earning reports tell(ad revenue up 12% YoY)
Post reply on HN