TOP500 at ISC’26: We have a New Number 1 Supercomputer

The new Chinese supercomputer beats all US supercomputers also in HPCG, not only in Linpack.

What is remarkable is that this was done despite the US attempts of sabotaging HPC in China by "sanctions".

This uses custom CPUs designed in China, which implement an Armv9-A ISA with SME (scalable matrix extension) and which use fast HBM memory. These CPUs are fast enough that they do not need any GPUs for exceeding the throughput of the American supercomputers, which use GPUs. This is like in the Japanese Fugaku, which was the first to implement the Armv8-A ISA with SVE, but which now is rather old.

Like in all CPU-based supercomputers, for this new Chinese supercomputer it is much easier to reach a higher percentage of the theoretical maximum throughput, when solving any problem. So for most practical problems it will be faster than a GPU-based supercomputer that would have the same theoretical maximum throughput.

So this is a much more interesting supercomputer than those built by just buying some HPC racks from HPE (Cray). Because China was forbidden to buy the American equipment, they had to innovate and design their own. Eventually they made something better than what they could not buy.

The AI oriented GPUs or TPUs have either weak FP64 throughput or they may not support FP64 at all.

They can compete neither with CPUs nor with GPUs that have good FP64 support, like the AMD CDNA datacenter GPUs, which occupy all the top places among American supercomputers.

NVIDIA has stopped improving the FP64 throughput even in their "datacenter" GPUs, abandoning this nowadays smaller market to AMD.

The AMD CDNA GPUs can be used for both HPC and AI, so only an AI cluster based on them could have dual use, but most who want AI choose NVIDIA.

The reason is that with GPUs it is far more difficult to reach a great percentage of the maximum theoretical throughput. Most GPU programs reach only a very small fraction of what is theoretically possible, and in the best cases one may reach something like 50% to 60% of the maximum.

This CPU-based supercomputer has demonstrated reaching 80% of the theoretical maximum throughput, and this is typical for CPU-based supercomputers. It is much easier to write efficient programs for CPUs.

The new custom Chinese CPUs, which use SME, the Arm Scalable Matrix Extension, are fast enough that they have beaten all GPU-based supercomputers, so there was no need to use GPUs.

Moreover these CPUs use HBM for a very fast memory interface, so in the benchmarks that depend more on memory bandwidth they have an even greater advance over the US GPU-based supercomputers. Thus there really was no point in using GPUs.

GPUs are necessary only when your CPUs are not good enough, which was not the case here.

In the recent past, the Japanese Fugaku used the same approach, of avoiding GPUs. At that time, their custom CPUs using the Armv8-A ISA with SVE were the first which used this ISA in HPC, but now that ISA variant is obsolete in comparison with the Armv9-A ISA with SME, which is implemented in these new custom Chinese CPUs.

Ironically, the related Graph500 benchmarks reflect this better. Performance is dependent more on using the hardware better than better hardware per se.