Live data from Hacker News

Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

tomshardware.com

141–150 of 182 posts

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#141
post #97
post #93

AMD should be gifting their GPUs by the dozens to the most prolific Open Source contributors if they want a piece of the cake. Their lack of access to CUDA is really harming them badly.

It's more than just that: for the money, their consumer GPUs don't compete in compute tasks (especially inference/training) and their Linux compute drivers are a pile of steaming garbage on consumer hardware. It's really interesting/depressing to watch as they've done a nice job of supplying good open source graphics drivers. They really seem to be lacking something at a leadership level in terms of understanding GPU…

I think that is underselling the big, slow push of their heterogeneous compute architecture. I don't understand the things, but as far as I can read it they've got 3.6GFLOP [0] GPU on those things as of 2022.

Nvidia are effortlessly crushing AMD right now and as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU (it is weirdly difficult to get a good tutorial on how to do matrix multiplication on an AMD GPU; every so often I look for one and have I think literally never found an example). But strategically, AMDs approach to GPU-CPU memory fusion is probably going to be the technically stronger approach. Assuming it works.

In hindsight they should have focused on libraries to let people use their GPU, but big picture they clearly understand how important it is to embrace general purpose compute and are treating it as a high priority.

[0] https://en.wikipedia.org/wiki/AMD_APU#Feature_overview

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#142
post #125
post #25

Technical detail: > Each A3 supercomputer is packed with 4th generation Intel Xeon Scalable processors backed by 2TB of DDR5-4800 memory. But the real "brains" of the operation come from the eight Nvidia H100 "Hopper" GPUs, which have access to 3.6 TBps of bisectional bandwidth by leveraging NVLink 4.0 and NVSwitch.

So it's not a super-computer, its a single server with 8 GPUs. Hilarious branding.

26 exaFlops sounds pretty super to me! My laptop only has 2.6 TFLOPs.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#143
post #141
post #97

Earlier quoted context omitted.

It's more than just that: for the money, their consumer GPUs don't compete in compute tasks (especially inference/training) and their Linux compute drivers are a pile of steaming garbage on consumer hardware. It's really interesting/depressing to watch as they've done a nice job of supplying good open source graphics drivers. They really seem to be lacking something at a leadership level in terms of understanding GPU…

I think that is underselling the big, slow push of their heterogeneous compute architecture. I don't understand the things, but as far as I can read it they've got 3.6GFLOP [0] GPU on those things as of 2022. Nvidia are effortlessly crushing AMD right now and as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU (it is weirdly difficult to get a good tutorial on how to do matrix mul…

> But strategically, AMDs approach to GPU-CPU memory fusion is probably going to be the technically stronger approach. Assuming it works.

I mean if anything, Nvidia is already there and crushing it too. CUDA has a unified memory model on Linux today and has for years, so if you have a proper pointer created by cudaMallocManaged, it can be used transparently in both GPU and CPU code without cudaMemcpy. And on the Grace Hopper chip, the open-source driver supports heterogeneous memory management, giving both the CPU and GPU unified, coherent memory across the CPU and GPU even though they have completely separate and isolated memory chips; 512GB LPDDR5X versus 96GB HBM3. This coherency is granular down to the cache line, too. So now every memory allocator and every system call and pointer can be passed directly to the GPU or from GPU to CPU freely.

And the open source driver supports HMM on normal x86_64/aarch64 Linux with consumer-level GPUs today, btw, but it's not as fast or granular. And then there are platforms like Jetson which have used single memory pools for a while; Orin uses a single shared bank of LPDDR5X chips for both CPU and GPU and will get HMM at some point in the future too I assume, though it uses a different driver.

Honestly the only place AMD seems to be winning in terms of compute is on large, bespoke contracts and features like unlocked FP64 performance with parts that are unobtanium and software stacks that have dedicated support engineers. Even Intel seems to be putting up more of a direct fight against Nvidia with oneAPI...

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#144
post #135

Earlier quoted context omitted.

If your demand for this is so urgent, it sounds like you want your own hardware. Here you go, that’ll be 38k for just the H100: https://serverevolution.com/nvidia-900-21010-0000-000.html

Does that come with the model weights? Not really relevant then to their announced products is it?

What model weights is the Google's A3 supercomputer supposed to come with? It's an announcement of new hardware available in GCP.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#145
post #77

Should we buy Nvidia stock then? The greatest technological advancement in recent years critically depends on the hardware from a single company with no competition. yet Nvidia stock is still below its 2021 peak. How so?

Because making decisions on account of an asset's price being higher 2 years ago is just falling victim to price anchoring? Would Nvidia not be worth buying in 2020 because its price was much lower in 2018 and thus must be overvalued in 2020? Investments should be based on the actual value of the company relative to its price, as well as relative to other investment oppertunities. Trying to making a profit by trading…

But the question isn't "can I do better than teams of quants who do this 100 hrs/wk and are supported by institutions with effectively infinity dollars", but "can I make money on this"? If I buy NVDA at 283, will it go up? There's no guarantee it will, they could lose their edge to AMD and the GPU market could bottom out, but barring some calamity, the answer seems to be yes they well. There maybe other stocks out there that are better buys, but they're part of the SP500 for a reason.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#146
post #143
post #141

Earlier quoted context omitted.

I think that is underselling the big, slow push of their heterogeneous compute architecture. I don't understand the things, but as far as I can read it they've got 3.6GFLOP [0] GPU on those things as of 2022. Nvidia are effortlessly crushing AMD right now and as far as I can tell it is because they implemented a bunch of BLAS functions on the GPU (it is weirdly difficult to get a good tutorial on how to do matrix mul…

> But strategically, AMDs approach to GPU-CPU memory fusion is probably going to be the technically stronger approach. Assuming it works. I mean if anything, Nvidia is already there and crushing it too. CUDA has a unified memory model on Linux today and has for years, so if you have a proper pointer created by cudaMallocManaged, it can be used transparently in both GPU and CPU code without cudaMemcpy. And on the Grac…

> And ... the Grace Hopper chip ... supports heterogeneous memory management

That is the point though, isn't it? Nvidia and AMD are converging to the same model, so it isn't fair to say AMD doesn't understand GPU compute. Nvidia just had a much neater implementation path where they hacked together something that worked in software while their hardware team figured out how to actually implement it. Technically it is arguable that they're behind AMD on general GPU compute, although that'd be pedantic given how thoroughly AMD failed to get their customers a place in the GPGPU market for the last decade.

AMD is floundering, no question. But the failure was understanding the path-dependent implementation aspects. They do understand that GPU compute is essential to the future of computing as an industry. They're clearly putting a lot of resources into that vision and they have been for around 20 years (similar timeline to CUDA).

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#147

It is interesting how the definition of a supercomputer changes over time. Compared to decades ago now everyone carries a supercomputer.

Apparently RTX4090 with FP8 is equivalent to worlds fastest supercomputer from 2007. So, in some sense I have a supercomputer on my desk:)

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#148
post #46

I'm more interested in what normal folks are running at home. What are your builds?

FYI rumour has it next round of titan GPUs are supposedly coming with 48GB Of course there is always something better on horizon, but if you're building soon that may be worth the wait

Yes please. Just hope it's not a 5-slot card.

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#149
post #71

Earlier quoted context omitted.

This is for GCP. Google themselves probably still trains on custom hardware but they don't offer their latest and greatest hardware on GCP. Offering more options to customers is always better especially when Nvidia has great market share in this area. This is probably the reason why Microsoft is trying to help AMD catch up so their is more competition. AI GPU prices are insane compared to standard GPU because of the…

I haven't heard anything about microsoft helping AMD, it sounds interesting. Do you mind linking an article?

It doesn't seems to be true.

There were articles that Microsoft was helping AMD, but the denied it.

https://arstechnica.com/gadgets/2023/05/microsoft-and-amd-ar...

Re: Google Launches AI Supercomputer Powered by Nvidia H100 GPUs

#150
post #137

Earlier quoted context omitted.

There are models available in GCP under Vertex AI category, I'm using the API to access them.

Oh that's great. Curious what the model sizes are but then to be fair gpt4 isn't publicly saying that either. Side smaller complaint - whats the point in these wait-lists if they never tell me when stuff actually launched.

These were available right at announcement time and not wait listed, I was using them while the keynote was still going on. The lists are for other products & wrappers around the foundational models.

I haven't seen otter or unicorn models, nor can I find tune them yet.

Post reply on HN