Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

nextplatform.com

Post reply on HN
Print view View original

11–20 of 39 posts

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#11

Post by baybal2 »

diamondlovesyou wrote: post #7
modeless wrote: post #2

4.9x faster FP64, but only 1.2x faster FP16. FP64 is totally irrelevant for machine learning. A100 has been available on AWS for over a year already, while MI200 won't be available at all until next year. Nvidia's A100 successor should be out next year too. And Nvidia's software stack puts AMD's to shame. AMD needs to do a lot better if they want to start being competitive in ML.

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't attempting to complete on f16 perf. They're completing where Nvidia's perf is abysmal: f64. Having programmed AMDGPUs at a lower level than HIP/Rocm, they are actually much better than Nvidia (and in fact I'm able to do cool things like pcie large bar/p2p even…

That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#12

Post by volta83 »

I wonder how they manage to keep the FP64 units busy. Seems this is an HPC product, but many HPC apps are memory bound. So to improve FP64 perf by 4 one might need to improve DRAM bandwidth by 8-16x. Otherwise the units would only be stalled waiting for memory.

But it seems they did not improve bandwidth by much?

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#13

Post by marmaduke »

modeless wrote: post #2

4.9x faster FP64, but only 1.2x faster FP16. FP64 is totally irrelevant for machine learning. A100 has been available on AWS for over a year already, while MI200 won't be available at all until next year. Nvidia's A100 successor should be out next year too. And Nvidia's software stack puts AMD's to shame. AMD needs to do a lot better if they want to start being competitive in ML.

It’s not irrelevant for machine learning in general, e.g. Stan (Bayesian/MCMC) is fp64 only, so its OpenCL offloading is more effective where fp64 is better supported.

It’s less relevant for the kinds of deep neural net work enabled by a vendor who’s software stack is designed around 32 and lower bit types, perhaps in an attempt to ensure their flops numbers are highest?

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#14

Post by dvdkhlng »

volta83 wrote: post #12

I wonder how they manage to keep the FP64 units busy. Seems this is an HPC product, but many HPC apps are memory bound. So to improve FP64 perf by 4 one might need to improve DRAM bandwidth by 8-16x. Otherwise the units would only be stalled waiting for memory. But it seems they did not improve bandwidth by much?

I don't know anything about the details here, but with usual linear algebra stuff, bandwidth depends on the size of the kernel that fits into the local memory inside whatever IC you use for your floating-point computation.

E.g. matrix multiplication of n×n square matrices has computational cost of n³ but bandwidth cost of n². Usuall a big m x m matrix is split into many blocks of n×n matrices (with m = k×n). If a n×n matrix fits into the local store of your CPU (cache or registers), then bandwidth cost for the m x m matrix product is k³×n×n = m×m×m/n, so the bigger the block-size 'n' that you can process inside the CPU, the less bandwidth you need.

edit: formatting

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#15

Post by mkl »

baybal2 wrote: post #11
diamondlovesyou wrote: post #7

Earlier quoted context omitted.

Machine learning isn't the only useful thing to run on "GPU"s (these can't do graphics workloads anymore; they're basically just vector processors). AMD isn't attempting to complete on f16 perf. They're completing where Nvidia's perf is abysmal: f64. Having programmed AMDGPUs at a lower level than HIP/Rocm, they are actually much better than Nvidia (and in fact I'm able to do cool things like pcie large bar/p2p even…

That's a smart move given GPUs may now loose a lot of market as "AI" thing is rapidly losing steam, and SLIDE is getting better, and better.

More on SLIDE for those curious: https://twimlai.com/slide-the-algorithmic-alternative-that-o...

It's a CPU-based deep learning training algorithm that beats GPUs for some tasks.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#16

Post by mhh__ »

atty wrote: post #4

I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their…

Not sure I really trust AMD not to trip over their own feet when it comes to software unfortunately.

Hats off to them if they manage to really land a blow on NV here.

Also potentially keep an eye on Intel a few years down the line? Since they can actually do software in my experience (and, get this AMD, document the software!)

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#17

Post by bayindirh »

atty wrote: post #4

I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning. I think that’s a smart thing on their part, because Nvidia really has a chokehold on the deep learning field right now with the A100 and CUDA/CUDNN software stack. I also find it interesting (and a good sign) that their…

> I find it interesting they’ve focused so much silicon on improving FP64 - in my mind that means they’re targeting physics simulations and other more traditional HPC workloads more than deep learning.

I develop scientific simulation software, and I can't tell you how happy I'm about it. Because while doing high precision work, GPUs fall flat fast.

I also work at a HPC center, and there's mountains of FP64 dependent applications running on CPUs. Moving them to GPUs will bring a lot of improvements in a lot of disciplines. It's not uncommon to let things run for a week on multiple nodes for meaningful results.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#18

Post by my123 »

volta83 wrote: post #12

I wonder how they manage to keep the FP64 units busy. Seems this is an HPC product, but many HPC apps are memory bound. So to improve FP64 perf by 4 one might need to improve DRAM bandwidth by 8-16x. Otherwise the units would only be stalled waiting for memory. But it seems they did not improve bandwidth by much?

> I wonder how they manage to keep the FP64 units busy

They don’t. See https://www.amd.com/en/graphics/server-accelerators-benchmar....

The MI250X, despite being dual big dies, doesn’t do especially well.

Re: Stacking Up AMD MI200 versus Nvidia A100 Compute Engines

#20

Post by freemint »

kolbusa wrote: post #10
baybal2 wrote: post #8

Earlier quoted context omitted.

Supercomputer market is still there. Intel Sapphire rapids owes its existence to the next US nuke design (to be made on Aurora supercomputer.)

I don't think so. The new ISA that is in the SPR is mostly about deep learning: it supports int8 and bfloat16 ( https://fuse.wikichip.org/news/3600/the-x86-advanced-matrix-... ). You can emulate higher precision using bfloat16 ( https://arxiv.org/abs/1904.06376 ), but I have not seen this used in the wild.

You know about surrogate models being run in simulations (physics informed neural network and all that stuff) the US is investing a lot in this. Maybe then it makes sense why bfloat16 exist in SPR.
Post reply on HN