Nvidia DGX GH200 Whitepaper
resources.nvidia.com
Nvidia DGX GH200 Whitepaper
1–10 of 45 posts
Re: Nvidia DGX GH200 Whitepaper
#2Why is this called a whitepaper, as this is more of a documentation and architecture overview of the cluster? Wow a CLOS topology for networking, very innovative.
Details on NVLink would be great. For example, the needs and problems solved by their custom cables seemingly required by NVLink would be worth a whitepaper.
Don't get me wrong, this is still great the general public can get a glimpse into Grace Hopper. And they do a good job of simplifying while throwing around mind-boggling numbers (the NVLink bandwidth is insane, though no words on latency, crucial for remote memory access).
Re: Nvidia DGX GH200 Whitepaper
#3The memory and bandwidth numbers are mind blowing. Going to be very hard to catch Nvidia. It’s as if competitors are going through the motions for participation prizes.
Re: Nvidia DGX GH200 Whitepaper
#4So basically 2x faster than H100
Re: Nvidia DGX GH200 Whitepaper
#5So basically 2x faster than H100
[deleted]
Re: Nvidia DGX GH200 Whitepaper
#6As context: 1x dgx gh200 has 256x gh200s which each have 1x h100 and 1x grace cpu
Re: Nvidia DGX GH200 Whitepaper
#7So basically 2x faster than H100
They claim 1.1x to 7x, depending on what you're doing. The 10% to 50% is for the ~10k GPU LLM training, where the main bottleneck tends to be networking:
> DGX GH200 enables more efficient parallel mapping and alleviates the networking communication bottleneck. As a result, up to 1.5x faster training time can be achieved over a DGX H100-based solution for LLM training at scale.
Re: Nvidia DGX GH200 Whitepaper
#8As context: 1x dgx gh200 has 256x gh200s which each have 1x h100 and 1x grace cpu
Adding up to "1 exaFLOPS" (sparse FP8). For reference, the fastest FP64 supercomputer is the AMD-based Frontier supercomputer, at 1.1 exaFLOPS.
Re: Nvidia DGX GH200 Whitepaper
#9I wonder how much this thing will cost, best I've been able to find so far is a 'low 8 digits' estimate in Anandtech article but nothing more specific than that.
https://www.anandtech.com/show/18877/nvidia-grace-hopper-has...
Re: Nvidia DGX GH200 Whitepaper
#10I would be interesting to know what kind of next-gen models this can train.
On the LLM frontier, we’re starting to hit the limits of reasoning abilities in the current gen.