Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia Ampere GA102 GPU Architecture [pdf]

nvidia.com

Post reply on HN
Print view View original

31–40 of 51 posts

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#31

Post by calo_star »

dragontamer wrote: post #5

I posted this a day or two ago: The A100 whitepaper "spoiled" a lot of these factoids already. ( https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent ...) The new bit seems to be the doubling of FP32 "CUDA cores" (I really hate that word: when Intel or AMD double their CPU pipelines it doesn't mean that they're selling more cores, it means their cores got wider... anyway). A100 didn't have this feature (I ass…

I think "parallel execution of fp32/int32" is kind of vaguely defined by them... Do they mean fp32/int32 instructions from the same thread (aka warp/wavefront) or from different threads? If it's the latter I'm pretty sure AMD GPUs have been doing it too.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#32

Post by Azrael3000 »

amelius wrote: post #2

These GPU architectures are too complicated for my taste. It reminds me of the days of segmented memory, and makes me feel like having to jump through hoops all the time. I'm curious if people have tried to invent something more elegant, and where these approaches have failed. Also, the fact that NVidia has probably patented this particular architecture makes it less interesting for me to really dig into.

I agree. It's a huge issue for software dev companies as it is related to a non-negligible effort to keep your software working efficiently with a large array of GPUs. One potential workaround are libraries such as KOKKOS [0]. Of course it's never going to be as fast as pure optimized CUDA, but if it gets you half the way there, you'll still be faster than using the CPU. Particularly when considering that some of the new CPUs seem to have scaling issues themselves [1]

[0] https://github.com/kokkos/kokkos [1] https://www.pugetsystems.com/labs/hpc/Threadripper-3990x-64-...

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#33

Post by Animats »

amelius wrote: post #2

These GPU architectures are too complicated for my taste. It reminds me of the days of segmented memory, and makes me feel like having to jump through hoops all the time. I'm curious if people have tried to invent something more elegant, and where these approaches have failed. Also, the fact that NVidia has probably patented this particular architecture makes it less interesting for me to really dig into.

Each of the internal units is reasonably simple compared to a modern superscalar x86. There are just huge numbers of units. The complexity is in the software needed to keep all those units busy and pumping around data between them. What you're doing down at the bottom of rendering or machine learning are usually very simple computations done a huge number of times. Somewhere above that is the problem of parceling out work to all those hardware resources in a somewhat optimal way. That's the hard problem.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#34

Post by pjmlp »

CyberDildonics wrote: post #15
modeless wrote: post #6

Earlier quoted context omitted.

It's a good question actually. Intel tried to make a GPU called Larrabee that was mostly a bunch of small x86 cores with giant vector units. Turns out that it couldn't compete in rendering performance on existing games (in 2010) without the fixed function units that GPUs have, so they canceled it as a GPU. It did result in the AVX-512 instruction set though. I think the idea still has promise but there's a chicken an…

They did actually make it and it was not supposed to be a GPU..

Sure it was, I was at the GDCE 2009 session on Larrabe.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#35

Post by genocidicbunny »

dragontamer wrote: post #5

I posted this a day or two ago: The A100 whitepaper "spoiled" a lot of these factoids already. ( https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent ...) The new bit seems to be the doubling of FP32 "CUDA cores" (I really hate that word: when Intel or AMD double their CPU pipelines it doesn't mean that they're selling more cores, it means their cores got wider... anyway). A100 didn't have this feature (I ass…

RTX I/O is going to be a big feature, and games are likely some of the first consumer-facing software that will use it because it is a standard features for the next console generation. And AAA devs already support multiple performance profiles, feature support fallbacks..etc. There's no reason they couldn't have the engine take advantage of RTX I/O when it exists, but otherwise fall back on an emulation layer of sorts.

In addition, I suspect the slice of the video game market that has a GPU with RTX I/O capability will also have a NVME SSD. Now, this is niche, but with that slice of the market also being the top-end performance tier, they're still going to be catered to by AAA devs.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#37

Post by formerly_proven »

dragontamer wrote: post #8
modeless wrote: post #6

Earlier quoted context omitted.

It's a good question actually. Intel tried to make a GPU called Larrabee that was mostly a bunch of small x86 cores with giant vector units. Turns out that it couldn't compete in rendering performance on existing games (in 2010) without the fixed function units that GPUs have, so they canceled it as a GPU. It did result in the AVX-512 instruction set though. I think the idea still has promise but there's a chicken an…

Fujitsu's A64FX ARM is proof that 512 bit SIMD can work on a CPU based platform.

For graphics use GPUs perform a very significant amount of work in hardware (rasterization and texture interpolation being the two computationally most intensive [probably followed by ROP, which blends pixel shader output into the framebufer]; you can easily calculate that the ALU bandwidth of the TMUs is about the same order of magnitude as all the shader cores), which gives them a huge efficiency lead over anything purely done with programmable hardware only.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#38

Post by Guthur »

jagger27 wrote: post #13
dragontamer wrote: post #5

I posted this a day or two ago: The A100 whitepaper "spoiled" a lot of these factoids already. ( https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent ...) The new bit seems to be the doubling of FP32 "CUDA cores" (I really hate that word: when Intel or AMD double their CPU pipelines it doesn't mean that they're selling more cores, it means their cores got wider... anyway). A100 didn't have this feature (I ass…

> The FP32 doubling, is one of the most important bits here. But fortunately for programmers, this doesn't really change how you do your code. Early benchmarks are showing games under-performing quite a bit in the worst cases. The crux of the issue is that it's not /exactly/ a no-compromise doubling of FP32. Each data path per SM can either do 2xFP32 or 1xINT32/1xFP32 per clock cycle. So if your game or application h…

I've been thinking that's why we are seeing the true doubling in full RTX like quake and minecraft but not on more traditional rendering engines.

From my understanding int is often used for lookups, and I'd presume a lot of that is some sort of environment mapping which adds some contention as int is more limited and "steals" from the doubling of FP.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#39

Post by dr_zoidberg »

genocidicbunny wrote: post #35
dragontamer wrote: post #5

I posted this a day or two ago: The A100 whitepaper "spoiled" a lot of these factoids already. ( https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent ...) The new bit seems to be the doubling of FP32 "CUDA cores" (I really hate that word: when Intel or AMD double their CPU pipelines it doesn't mean that they're selling more cores, it means their cores got wider... anyway). A100 didn't have this feature (I ass…

RTX I/O is going to be a big feature, and games are likely some of the first consumer-facing software that will use it because it is a standard features for the next console generation. And AAA devs already support multiple performance profiles, feature support fallbacks..etc. There's no reason they couldn't have the engine take advantage of RTX I/O when it exists, but otherwise fall back on an emulation layer of sor…

Even without an NVMe drive you're better off with this just by skypping system RAM altogether. Bu you're not going to be able to use it to stream back and forth game content at the snap of a finger (well maybe that's a bit hyperbolic) as the console makers have been saying they will.

Re: Nvidia Ampere GA102 GPU Architecture [pdf]

#40

Post by dr_zoidberg »

formerly_proven wrote: post #37
dragontamer wrote: post #8

Earlier quoted context omitted.

Fujitsu's A64FX ARM is proof that 512 bit SIMD can work on a CPU based platform.

For graphics use GPUs perform a very significant amount of work in hardware (rasterization and texture interpolation being the two computationally most intensive [probably followed by ROP, which blends pixel shader output into the framebufer]; you can easily calculate that the ALU bandwidth of the TMUs is about the same order of magnitude as all the shader cores), which gives them a huge efficiency lead over anything…

Michael Abrash had a great series of articles in Dr. Dobbs detailing how he came to work for Intel (which spun Larabee) after talking at a game conference with some of their people to ask them for a lerp (linear interpolation) instruction in x86 extensions[0] :)

Oh and Larabee gave us more than AVX512, it also gave us the Xeon Phis, which were accelerators (much akin to the GPGPU of nvidia GPUs?) aimed at scientific code undeer the promise that "since it's x86, you don't need to change your code that much!". However:

> An empirical performance and programmability study has been performed by researchers, in which the authors claim that achieving high performance with Xeon Phi still needs help from programmers and that merely relying on compilers with traditional programming models is still far from reality. However, research in various domains, such as life sciences, and deep learning demonstrated that exploiting both the thread- and SIMD-parallelism of Xeon Phi achieves significant speed-ups.

(from Wikipedia[1])

So pretty much the same as a GPU. It is a bit unfortunate that, in theory, good OpenCL support could have made running this code in 2/4/8 core CPUs (with or without SMT) or in the thread-beast that are/were the Phis. But that woud've probably required OpenCL to be a bit more mature, and Intel skipped that train too.

[0] https://www.drdobbs.com/parallel/a-first-look-at-the-larrabe...

[1] https://en.wikipedia.org/wiki/Xeon_Phi

Post reply on HN