Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Japan Captures TOP500 Crown with Arm-Powered Supercomputer

top500.org

Post reply on HN
Print view View original

11–20 of 220 posts

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#11

Post by blopeur »

sadfev wrote: post #6

Can any of the HPC experts shed some light on how these ARM chips are better than their predecessors. I toured a small cluster in LANL, where the ARM chips ran the hottest and their cooling was the loudest.

A64fx have on board HMB -> that means no dram. If you look at the fugaku mother board their is no Dimm slots. All the memory is on the same package as the CPU.

This delivers a huge boost in bandwith.

HMB stand for high memory bandwidth. It offers up to 900 GB/s.

Now if you add the tofu interconnect on top you have a systems finely tuned for maximising data movement.

Remember : compute is cheap, communication is expensive.

You can have load of gpu and processors but if you can't feed them data fast enough they are useless.

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#13

Post by ViralBShah »

fhqghds wrote: post #7
ViralBShah wrote: post #3

Given that today's HPC architectures are mostly power constrained, and a majority of the FLOPS often come from GPUs (for their flop/watt ratios), this direction is not surprising. ARM has been making major strides in the high performance area. The new AWS Graviton processors are pretty nice from what I have heard. And then there's the ARM in Mac. Yup and Julia will run on all of these! While I say all of this, I shou…

Get ready for a surprise then: all those FLOPS are coming from the ARM cores.... This beast has no GPUs: https://postk-web.r-ccs.riken.jp/spec.html

That's pretty cool! That probably means that applications will have an easier time. Looks like it has 512-bit SIMD.

I wonder what BLAS they are using, and if the contributions are open sourced.

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#14

Post by fhqghds »

xhkkffbf wrote: post #8

Not to be dismissive, but can't anyone "build" the biggest supercomputer by reserving enough instances at AWS or GCP? I'm sure that AWS or GCP would like to encourage this competition, but it seems a bit, well, boring.

yyyeah... no.

A major part of what makes these machines special is their interconnect. Fujitsu is running a 6D torus interconnect with latencies well in the sub-usecond range. The special sauce is ability of cores to interact with each other with extreme bandwidth at extremely low latencies.

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#15

Post by ViralBShah »

blopeur wrote: post #11
sadfev wrote: post #6

Can any of the HPC experts shed some light on how these ARM chips are better than their predecessors. I toured a small cluster in LANL, where the ARM chips ran the hottest and their cooling was the loudest.

A64fx have on board HMB -> that means no dram. If you look at the fugaku mother board their is no Dimm slots. All the memory is on the same package as the CPU. This delivers a huge boost in bandwith. HMB stand for high memory bandwidth. It offers up to 900 GB/s. Now if you add the tofu interconnect on top you have a systems finely tuned for maximising data movement. Remember : compute is cheap, communication is expen…

That is a pretty fun architecture. I hope that opens the door to higher performance for more workloads than top500.

At least with the top500 benchmark, the bandwidth is not a problem, so long as you can do a large enough problem. Since it is a linear solve that spends all its time doing matmul (n^3 operations on n^2 data), so long as the problem is big enough, you can saturate the cores.

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#16

Post by floren »

xhkkffbf wrote: post #8

Not to be dismissive, but can't anyone "build" the biggest supercomputer by reserving enough instances at AWS or GCP? I'm sure that AWS or GCP would like to encourage this competition, but it seems a bit, well, boring.

The ranking is calculated based on the Linpack benchmark. Being a parallel application, performance is not simply scaled to number of processors; the network interconnect is hugely important.

Now, although Linpack is a better evaluation metric for a supercomputer than simply totaling up # of processors and RAM size, it's still a very specific benchmark of questionable real-world utility; people like it because it gives you a score, and that score lets you measure dick-size, err, computing power. It also, if you're feeling unscrupulous, lets you build a big worthless Linpack-solving machine which generates a good score but isn't as good for real use (an uncharitable person might put Roadrunner https://en.wikipedia.org/wiki/Roadrunner_(supercomputer) in this category)

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#17

Post by mxcrossb »

xhkkffbf wrote: post #8

Not to be dismissive, but can't anyone "build" the biggest supercomputer by reserving enough instances at AWS or GCP? I'm sure that AWS or GCP would like to encourage this competition, but it seems a bit, well, boring.

AWS has made it into the top500 a few times in fact, though not that high on the list. I think the main issue would be reserving enough machines that have a high performance network between them, which is not a typical cloud need.

But the more interesting question for me is: on an embarrassingly parallel workload, how does Amazon’s full infrastructure compare to these top machines? I’d assume that Amazon keeps that a secret.

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#18

Post by wenc »

ViralBShah wrote: post #3

Given that today's HPC architectures are mostly power constrained, and a majority of the FLOPS often come from GPUs (for their flop/watt ratios), this direction is not surprising. ARM has been making major strides in the high performance area. The new AWS Graviton processors are pretty nice from what I have heard. And then there's the ARM in Mac. Yup and Julia will run on all of these! While I say all of this, I shou…

> based on your ability to solve the largest dense linear solve you possibly can - something almost no real application does.

Sounds right.

I was going to say what about large-scale optimization problems? But I realized that most typically only require sparse linear solves.

Gradient descent does require the solution of dense Ax=b systems. But the most visible/popular application of large-scale gradient descent today, neural networks, typically use SGD which require no dense linear solves at all.

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#19

Post by lukevp »

blopeur wrote: post #11
sadfev wrote: post #6

Can any of the HPC experts shed some light on how these ARM chips are better than their predecessors. I toured a small cluster in LANL, where the ARM chips ran the hottest and their cooling was the loudest.

A64fx have on board HMB -> that means no dram. If you look at the fugaku mother board their is no Dimm slots. All the memory is on the same package as the CPU. This delivers a huge boost in bandwith. HMB stand for high memory bandwidth. It offers up to 900 GB/s. Now if you add the tofu interconnect on top you have a systems finely tuned for maximising data movement. Remember : compute is cheap, communication is expen…

That's fascinating. I know that AMD has been touting HBM as a faster memory subsystem for their GPUs, is that the same as HMB where it's stacked? Or are they just calling it something similar?

Re: Japan Captures TOP500 Crown with Arm-Powered Supercomputer

#20

Post by pininja »

The link is working for me, here are the details on the winner.

“The new top system, Fugaku, turned in a High Performance Linpack (HPL) result of 415.5 petaflops, besting the now second-place Summit system by a factor of 2.8x. Fugaku, is powered by Fujitsu’s 48-core A64FX SoC, becoming the first number one system on the list to be powered by ARM processors. In single or further reduced precision, which are often used in machine learning and AI applications, Fugaku’s peak performance is over 1,000 petaflops (1 exaflops). The new system is installed at RIKEN Center for Computational Science (R-CCS) in Kobe, Japan.

Post reply on HN