Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

nextplatform.com

Post reply on HN
Print view View original

21–30 of 39 posts

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#21

Post by freemint »

baybal2 wrote: post #11
diamondlovesyou wrote: post #7

Earlier quoted context omitted.

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't attempting to complete on f16 perf. They're completing where Nvidia's perf is abysmal: f64. Having programmed AMDGPUs at a lower level than HIP/Rocm, they are actually much better than Nvidia (and in fact I'm able to do cool things like pcie large bar/p2p even…

That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.

I've not seen anything convincing that slide can approximate convolutional layers like at all without almost doing a full convolution at which point it is not competitive.

Can you link me to paper about recent (2021?) work or improvements with slide?

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#22

Post by cnschn »

diamondlovesyou wrote: post #7
modeless wrote: post #2

4.9x faster FP64, but only 1.2x faster FP16. FP64 is totally irrelevant for machine learning. A100 has been available on AWS for over a year already, while MI200 won't be available at all until next year. Nvidia's A100 successor should be out next year too. And Nvidia's software stack puts AMD's to shame. AMD needs to do a lot better if they want to start being competitive in ML.

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't attempting to complete on f16 perf. They're completing where Nvidia's perf is abysmal: f64. Having programmed AMDGPUs at a lower level than HIP/Rocm, they are actually much better than Nvidia (and in fact I'm able to do cool things like pcie large bar/p2p even…

Just curious, what lower level options are there? Inline assembly in OpenCL?

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#23

Post by fxtentacle »

freemint wrote: post #21
baybal2 wrote: post #11

Earlier quoted context omitted.

That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.

I've not seen anything convincing that slide can approximate convolutional layers like at all without almost doing a full convolution at which point it is not competitive. Can you link me to paper about recent (2021?) work or improvements with slide?

I thought SLIDE was optimized for classification with mind-boggingly sparse data. I.e. "choose the best label for this image out of those 1 mio choices".

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#24

Post by pixelpoet »

bayindirh wrote: post #17
atty wrote: post #4

I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their…

> I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I develop scientific simulation software, and I can't tell you how happy I'm about it. Because while doing high precision work, GPUs fall flat fast. I also work at a HPC center, and there's mountains of FP64 dependent…

If I could in any way afford it I'd buy one of these just for doing (hobby) n-body simulations and fractal rendering. I bought a Radeon VII specifically for this and it died a month or so after I got it, with no replacement possible because they were EOL :( So badly want a replacement, but with GPU prices as they are now, there's just no way to justify buying a 2nd hand one.

Right now I think the best you can do for FP64/$ is AVX-512 CPUs and Radeon GPUs.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#25

Post by arcanus »

my123 wrote: post #18
volta83 wrote: post #12

I wonder how they manage to keep the FP64 units busy. Seems this is an HPC product, but many HPC apps are memory bound. So to improve FP64 perf by 4 one might need to improve DRAM bandwidth by 8-16x. Otherwise the units would only be stalled waiting for memory. But it seems they did not improve bandwidth by much?

> I wonder how they manage to keep the FP64 units busy They don’t. See https://www.amd.com/en/graphics/server-accelerators-benchmar... . The MI250X, despite being dual big dies, doesn’t do especially well.

I disagree. The website you linked to shows speed-ups on MI250X between 1.6x and 3x higher than A100. The theoretical memory bandwidth speed-up between MI250X and A100 is only 1.6X (3.2 TB/s vs 2.0 TB/s). Thus, I'd say they are seeing the advantage of higher FP64 compute in those applications.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#26

Post by homerowilson »

freemint wrote: post #21
baybal2 wrote: post #11

Earlier quoted context omitted.

That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.

I've not seen anything convincing that slide can approximate convolutional layers like at all without almost doing a full convolution at which point it is not competitive. Can you link me to paper about recent (2021?) work or improvements with slide?

The current work

https://proceedings.mlsys.org/paper/2021/hash/3636638817772e...

seems mostly about tuning the original idea instead of expanding its scope. But it's still a neat idea. I guess it could be possible to adpt many of the approximations used in the SLIDE idea to GPUs too though...

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#27

Post by Retric »

atty wrote: post #4

I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their…

Nvidia can’t improve FP64 performance to near parity without sacrificing a lot in the process, which would make them more vulnerable on the deep learning side of things.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#28

Post by dragontamer »

cnschn wrote: post #22
diamondlovesyou wrote: post #7

Earlier quoted context omitted.

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't attempting to complete on f16 perf. They're completing where Nvidia's perf is abysmal: f64. Having programmed AMDGPUs at a lower level than HIP/Rocm, they are actually much better than Nvidia (and in fact I'm able to do cool things like pcie large bar/p2p even…

Just curious, what lower level options are there? Inline assembly in OpenCL?

I never touched it myself, but AMD once exposed the "HSA" interface, which is the AMD GPU execution engine.

ROCm / HIP and OpenCL are built on top of that HSA level. I don't think any docs exist for it, but you can see a ton of references to HSA stuff if you browse the ROCm source code.

--------

It sounds like the parent post discusses details about how AMD GPUs "pick" the next kernel to run. I've been told that the AMD GPU is very advanced at this, but no adequate interface has ever been exposed to the programmer.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#29

Post by hyperbovine »

Retric wrote: post #27
atty wrote: post #4

I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their…

Nvidia can’t improve FP64 performance to near parity without sacrificing a lot in the process, which would make them more vulnerable on the deep learning side of things.

?? Build two separate product lines.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#30

Post by Retric »

hyperbovine wrote: post #29
Retric wrote: post #27

Earlier quoted context omitted.

Nvidia can’t improve FP64 performance to near parity without sacrificing a lot in the process, which would make them more vulnerable on the deep learning side of things.

?? Build two separate product lines.

With which engineers, managers, etc? Intel has long faced the same issues, they want their best R&D on x86 CPU’s so they had major issues expanding into other product lines.
Post reply on HN