Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

olcf.ornl.gov

Post reply on HN
Print view View original

11–20 of 28 posts

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#11

Post by tgamblin »

petschge wrote: post #6
gigatexal wrote: post #2

If I recall correctly the AMD hardware is competitive or better in raw compute than their nVidia counterparts but Cuda makes developing software so easy so I’m curious what ORNL’s take on that is.

I don't work for ORNL and don't know what their take in this is, but a lot of codes that the DoE uses are being ported to the Kokkos framework out of the Sandia National Lab. The application developer at that point basically doesn't have to care if the code runs on KNL, on Nvidia cards using Cuda or on AMD cards using ROCm. The work of tuning Kokkos well on AMD cards is cheap compared to getting stuck on a single ven…

At LLNL the story is similar, but the framework is RAJA: https://github.com/llnl/raja. RAJA is a bit simpler than Kokkos in that it does not require you to adopt its data structures.

The Kokkos and RAJA teams are also working together on some common utility libraries now.

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#12

Post by tgamblin »

vortico wrote: post #3
gigatexal wrote: post #2

If I recall correctly the AMD hardware is competitive or better in raw compute than their nVidia counterparts but Cuda makes developing software so easy so I’m curious what ORNL’s take on that is.

I'm wondering the same. I'd guess that 99% of science code that uses the GPU now depends on CUDA. The other 1% use other third-party GPU frontends which might also be tuned for Nvidia GPUs. I feel that it's good to use the AMD ecosystem to mix up the competition a bit, and the majority of other clusters will still use Nvidia if your application requires it, but I'm just not aware of any momentum at all (in the form o…

On the NNSA side of DOE, the codes very intentionally do not depend on CUDA. They use RAJA (https://github.com/llnl/raja) or Kokkos (https://github.com/kokkos/kokkos) to ensure that they're not tied to a particular GPU. Using those frameworks you can easily switch between CUDA, OpenMP, or whatever in a single-source codebase.

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#13

Post by minxomat »

rl3 wrote: post #4
vortico wrote: post #3

Earlier quoted context omitted.

I'm wondering the same. I'd guess that 99% of science code that uses the GPU now depends on CUDA. The other 1% use other third-party GPU frontends which might also be tuned for Nvidia GPUs. I feel that it's good to use the AMD ecosystem to mix up the competition a bit, and the majority of other clusters will still use Nvidia if your application requires it, but I'm just not aware of any momentum at all (in the form o…

> ... so library-writers will have to basically start from scratch if this AMD cluster is confirmed. I doubt library writers will care beyond anyone involved in the project. That said, it's possible to port CUDA code already with a bit of work, so not quite from scratch in any case: https://github.com/ROCm-Developer-Tools/HIP

See also PlaidML

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#14

Post by noir_lord »

gigatexal wrote: post #2

If I recall correctly the AMD hardware is competitive or better in raw compute than their nVidia counterparts but Cuda makes developing software so easy so I’m curious what ORNL’s take on that is.

The Radeon VII is, all though it's AMD's high end 'gaming' card the suspicioun is that was a secondary use and they shoved it out to have something competitive with the higher end nvidia cards (for gaming nvidia holds the edge, my 2080 is about 5-7% faster than a VII but the VII has twice the RAM and it's faster RAM).

> When looking at the geometric mean of all the OpenCL benchmarks carried out, the Radeon VII was 12% faster than the GeForce RTX 2080...

https://www.phoronix.com/scan.php?page=article&item=radeon-v...

It gets crushed by the Titans but then they cost massively more (more than double).

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#15

Post by throw20102010 »

minxomat wrote: post #13
rl3 wrote: post #4

Earlier quoted context omitted.

> ... so library-writers will have to basically start from scratch if this AMD cluster is confirmed. I doubt library writers will care beyond anyone involved in the project. That said, it's possible to port CUDA code already with a bit of work, so not quite from scratch in any case: https://github.com/ROCm-Developer-Tools/HIP

See also PlaidML

PlaidML is not what most people want. PlaidML is pretty much only for neural network based machine learning, plus it's primarily a backend for models written in Keras. But it is an option for the narrow use case of neural networks.

Contrary to what buzzword happy SV types would have you believe, most "real" HPC work isn't machine learning (r/gatekeeping, I know). Particle physics, computational fluid dynamics, network simulations, etc. Lots of it is already written in CUDA. HIP, using hipify, can translate the already written CUDA code to HIP, which is GPU-agnostic.

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#16

Post by simonbyrne »

hydroreadsstuff wrote: post #10

Here is a recent training from AMD for Frontier: Link: https://www.exascaleproject.org/event/amd-gpuprogramming-hip... Video: https://youtu.be/3ZXbRJVvgJs Slides: https://www.exascaleproject.org/wp-content/uploads/2017/05/O... It will be very interesting to see what AMD came up with to convince ORNL to switch from 7 years (or more?) of NVIDIA to something else. I don't think it's just a lower price. Perhaps AMD is do…

I seem to recall that part of the DoE's selection criteria is to ensure a competitive market for future contracts (since they will always be buying more supercomputers). Given the recent dominance of Nvidia in the market (see https://www.top500.org/lists/2019/06/), it probably made sense for them to ensure some contracts went to other suppliers (AMD for Frontier, Intel for Aurora) to ensure that Nvidia doesn't establish a monopoly on future tech.

Additionally, these sorts of supercomputers are also a way for governments to implicitly subsidise their tech industries: when viewed through that lens, spreading these contracts around makes a lot more sense.

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#17

Post by desertrider12 »

petschge wrote: post #9
gigatexal wrote: post #8

Earlier quoted context omitted.

Is it open source? It’d be huge to have something compete with cuda so that the best hardware may win

Kokkos (as in the C++ template frame work and a lot of tools to get insights into bottle necks) is opens source. See https://github.com/kokkos/kokkos and https://github.com/kokkos/kokkos-tools . They take pull requests and react to bug reports. They even plan to upstream some things from kokkos (such as multi-dimensional arrays) into future C++ standards. The mission related codes based on kokkos are very much not op…

There are some non-weapons codes like Albany and NaluCFD based on Kokkos and Trilinos that are open source.

[1] https://github.com/SNLComputation/Albany

[2] https://github.com/NaluCFD/Nalu

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#18

Post by jcranmer »

gigatexal wrote: post #2

If I recall correctly the AMD hardware is competitive or better in raw compute than their nVidia counterparts but Cuda makes developing software so easy so I’m curious what ORNL’s take on that is.

A shocking fact I heard recently is that on these large GPGPU supercomputers, only about 10% of the applications actually use the GPU at all.

The labs are not likely to be a big fan of CUDA, since the reality is that most scientists do not have the bandwidth to rewrite their software (even to use GPGPUs in the first place, see above), and Nvidia tries very hard to make sure that CUDA is impossible to use for other GPU vendors. The labs have a requirement to source from multiple vendors, so the CUDA lock-in is not something they are thrilled about.

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#19

Post by minxomat »

throw20102010 wrote: post #15
minxomat wrote: post #13

Earlier quoted context omitted.

See also PlaidML

PlaidML is not what most people want. PlaidML is pretty much only for neural network based machine learning, plus it's primarily a backend for models written in Keras. But it is an option for the narrow use case of neural networks. Contrary to what buzzword happy SV types would have you believe, most "real" HPC work isn't machine learning (r/gatekeeping, I know). Particle physics, computational fluid dynamics, networ…

Plaid is just one of 01org's projects. There are dozens of other projects (and compilers), which when combined can be used to build very intricate HPC applications and pipelines. Though for some core projects, most of the tuning development is only available on Intel platforms.

Re: Frontier: ORNL's 2021 exascale supercomputer will run on AMD CPUs and GPUs

#20

Post by throw20102010 »

minxomat wrote: post #19
throw20102010 wrote: post #15

Earlier quoted context omitted.

PlaidML is not what most people want. PlaidML is pretty much only for neural network based machine learning, plus it's primarily a backend for models written in Keras. But it is an option for the narrow use case of neural networks. Contrary to what buzzword happy SV types would have you believe, most "real" HPC work isn't machine learning (r/gatekeeping, I know). Particle physics, computational fluid dynamics, networ…

Plaid is just one of 01org's projects. There are dozens of other projects (and compilers), which when combined can be used to build very intricate HPC applications and pipelines. Though for some core projects, most of the tuning development is only available on Intel platforms.

1. Your previous comment only mentioned PlaidML.

2. The parent comment was about translating existing code written in CUDA to be used on an AMD GPU, not about developing new software- if it were about developing new software, they'd be starting from scratch with 01.org tools, which is what everyone wants to avoid. 01.org doesn't have any translation tools for this.

Post reply on HN