1 petaflop on a chip?? What is the catch?
The catch is it's only for TF32 computations (Nvidia proprietary 19 bit floating point number format)
Nvidia Hopper GPU Architecture and H100 Accelerator
51–60 of 183 posts
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#52Off topic but I can't stand when corporations use actual people's names for their marketing who never gave them the permission to do so. For something like Shakespeare or Cicero I'm OK with it but Grace Hopper was alive in my lifetime, and even Tesla feels a little weird. What gives you the right to use that person's reputation to shill your product?
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#53This seems fast... TF32 ....... 1,000 TFLOPS (tensor core) FP64/FP32 ... 60 TFLOPS I am more interested in the 144-core Grace CPU Superchip. nVidia is getting into the CPU business...
50% sparsity and rated at 700W. The new DGX is 10kW!
I was recently researching how you'd host systems like this in a datacentre and was blown away to find out that you can cool 40kW in a single air cooled rack - this might be old news for many, but it was 2x or 3x what I expected! Glad I'm not paying the electricity bill :)
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#54Earlier quoted context omitted.
That's a strange statement. The vast vast majority of these cards will be in systems running Linux.
I for one suffer deeply when I try to install the nvidia drivers on Linux. The website binaries _always_ break my system Only the ppas from graphics-drivers work properly My experience on windows is much more automatic and it never breaks anything. But I'd rather pay the price (installing on Linux) to avoid windows at all costs
If you installed the drivers using the PPAs, you can't then update using the NVIDIA-provided binaries without doing a very thorough purge, including deleting all dependent installs (CUDNN, CUBLAS, etc.)
I highly recommend sticking with one technique or the other; never intermix them.
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#55Sounds like we need some new training methods. If training could take place locally and asynchronously instead of globally through backpropagation, the amount of energy could probably be significantly reduced.
Trying to reduce energy consumption for ML like this is so silly.
Training costs are growing exponentially bigger.
The degree to which energy and capital costs can be optimized will determine how large they can go.
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#5680 billion transistors boggles my mind. How many molecules are their per transistor?
wat
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#57Off topic but I can't stand when corporations use actual people's names for their marketing who never gave them the permission to do so. For something like Shakespeare or Cicero I'm OK with it but Grace Hopper was alive in my lifetime, and even Tesla feels a little weird. What gives you the right to use that person's reputation to shill your product?
> What gives you the right to use that person's reputation to shill your product?
Practically speaking you have the right to do anything unless someone complains about it. A lot of popular figures, even those long dead, have estates and organizations that manage their likeliness and other related copyright and IP. IDK what the situation is in this case, but Nvidia may very well have paid for the name.
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#58Earlier quoted context omitted.
50% sparsity and rated at 700W. The new DGX is 10kW!
I was recently researching how you'd host systems like this in a datacentre and was blown away to find out that you can cool 40kW in a single air cooled rack - this might be old news for many, but it was 2x or 3x what I expected! Glad I'm not paying the electricity bill :)
Here's what a propane heater of similar output looks like: https://www.amazon.com/Dura-Heat-Propane-Forced-Heater/dp/B0...
Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#59Re: Nvidia Hopper GPU Architecture and H100 Accelerator
#60This would be cool if they had decent drivers for Linux.