Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

tomshardware.com

Post reply on HN
Print view View original

161–170 of 182 posts

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#161

Post by snek_case »

ttul wrote: post #26
IanCal wrote: post #9

Kinda feels like the main thing google launches is waiting lists.

And my main worry is: are they just going to cancel the new thing that my company invested six months and $250,000 of engineering time integrating with…

Probably doesn't matter that much because it's just hardware. Presumably not that hard to run your software on another intel box with nvidia GPUs. There's also plenty of demand for nvidia GPUs right now, still no guarantee given it's Google, but it would be hard not to make money with this.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#162

Post by TechnicolorByte »

ironbound wrote: post #132
noogle wrote: post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

Sure till Nvidia's lunch is eaten by hardware AI companies https://www.cerebras.net/andromeda/ https://tenstorrent.com/grayskull/

This narrative has been pushed for several years now with the likes of Habana, Cerebras, SambaNova, Graphcore, Tesla Dojo, etc.

And yet none of them seem to have made any dent in Nvidia’s dominance. None of them have any real presence on industry-standard MLPerf benchmarks (not even TPU releases all benchmarks and they started the damn benchmark).

The truth is that making an AI chip isn’t as simple as putting a bunch of matmuls together in a custom ASIC and pointing a driver at it; there’s hard work and optimization the entire stack down, many of which aren’t even focused on the math part.

So while I don’t doubt that some competitors (AMD?) will gain decent market share eventually, Nvidia’s probably not going to be displaced so easily.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#163

Post by gyrovagueGeist »

roenxi wrote: post #141
blihp wrote: post #97

Earlier quoted context omitted.

It's more than just that: for the money, their consumer GPUs don't compete in compute tasks (especially inference/training) and their Linux compute drivers are a pile of steaming garbage on consumer hardware. It's really interesting/depressing to watch as they've done a nice job of supplying good open source graphics drivers. They really seem to be lacking something at a leadership level in terms of understanding GPU…

I think that is underselling the big, slow push of their heterogeneous compute architecture. I don't understand the things, but as far as I can read it they've got 3.6GFLOP [0] GPU on those things as of 2022. Nvidia are effortlessly crushing AMD right now and as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU (it is weirdly difficult to get a good tutorial on how to do matrix mul…

> as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU

rocBLAS and other vendor agnostic numeric libraries have made a lot of progress in the past 2 years (mostly as a result of the DoE's exascale computing project)

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#164

Post by onion2k »

jsnell wrote: post #87
onion2k wrote: post #56

Earlier quoted context omitted.

The word TPU isn't even mentioned here. A sentence that reads "I am going to eat nothing but vegetables from now on" doesn't mention meat, but you can infer that I won't eat meat again from the sentence. A sentence that says Google are going all in on nVidea GPUs for AI doesn't need to mention TPUs to convey information about their future either.

Sure. That's why I mentioned multiple ways in which the article could have been relevant to TPUs, which you chose not to quote. But it didn't have any of those either. The sentence you're offering up as a demonstration is just something you made up that does not appear in the article. If anything, this just reinforces the point I was making. There is nothing at all in the article supporting this narrative. So, where…

At the very least we know that there's a team in Google that chose to build an AI supercomputer with non-Google hardware. They didn't, or wouldn't, work with the TPU team to do it, or they did and the TPU team couldn't get it to work. Or they could but something still made nVidia hardware more compelling. Every level of management involved were persuaded that this was the case even knowing it would send a message to people outside of Google about TPUs.

Andnfrom.all that we're meant to say it infers nothing about TPUs?

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#166

Post by roenxi »

gyrovagueGeist wrote: post #163
roenxi wrote: post #141

Earlier quoted context omitted.

I think that is underselling the big, slow push of their heterogeneous compute architecture. I don't understand the things, but as far as I can read it they've got 3.6GFLOP [0] GPU on those things as of 2022. Nvidia are effortlessly crushing AMD right now and as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU (it is weirdly difficult to get a good tutorial on how to do matrix mul…

> as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU rocBLAS and other vendor agnostic numeric libraries have made a lot of progress in the past 2 years (mostly as a result of the DoE's exascale computing project)

If progress means going from nothing however long ago to hardlocking my system today then progress achieved. But for me rocBLAS has not yet reached the lofty peaks of multiplying matricies together.

In fairness, my graphics card isn't supported - multiplying matricies being one of those advanced features that they only implemented in the last couple of years. Older graphics cards maybe don't have the grunt for that.

I love AMD, the linux graphics drivers are great. But their GPGPU platform is not good.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#167

Post by ukuina »

IanCal wrote: post #81
cloudking wrote: post #70

Earlier quoted context omitted.

Sure, but look at Bard it was on wait-list for what 3 months? Now it's available in 180 countries... for free. Not everything gets launched because sometimes they find out in that testing period that they got it wrong.

After the big hype of it yes. And the models are not really available, they've got a little playground for some unspecified model.

And when it underperforms relative to expectations, they claim it's not running the best model they have.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#168

Post by ukuina »

ttul wrote: post #26
IanCal wrote: post #9

Kinda feels like the main thing google launches is waiting lists.

And my main worry is: are they just going to cancel the new thing that my company invested six months and $250,000 of engineering time integrating with…

This is the way.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#169

Post by gyrovagueGeist »

roenxi wrote: post #166
gyrovagueGeist wrote: post #163

Earlier quoted context omitted.

> as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU rocBLAS and other vendor agnostic numeric libraries have made a lot of progress in the past 2 years (mostly as a result of the DoE's exascale computing project)

If progress means going from nothing however long ago to hardlocking my system today then progress achieved. But for me rocBLAS has not yet reached the lofty peaks of multiplying matricies together. In fairness, my graphics card isn't supported - multiplying matricies being one of those advanced features that they only implemented in the last couple of years. Older graphics cards maybe don't have the grunt for that.…

I’ve only used it on MI200 series cards but both direct API in C and through a cupy interface for matrix mul and triangular solves it’s worked well for me. There was a bit of bugginess for running on non-default streams a few months back, but seems fixed now.

Now AMDs platform for debugging and profiling GPGPUs apps on the other hand is a different story/mess and very very behind NVIDIAs solutions.

For sure the lack of consumer card support is annoying, all effort seems focused on satisfying their contracts and not expanding support into the much wider GPGPU market rn and I wish it wasn’t. It feels like an afterthought at times. I just wanna be able to compile and play around with HIP on my home computer, but :(

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#170

Post by joshlk »

lordofgibbons wrote: post #38

Does this mean Google is giving up on TPUs? TPUs were supposed to be their unfair advantage in the cloud ML/DL space. But from what I've experienced, and have heard from other engineers, there's always some subtle incompatibility with TPUs that requires modifying the training/eval scripts. I wonder why they didn't try to polish the rough edges with Pytorch, et al. If they're admitting TPUs aren't their competitive ad…

> subtle incompatibility with TPUs that requires modifying the training/eval scripts

Do you have any more details or links to articles expanding on this?

Post reply on HN