Nvidia CEO Introduces Nvidia Ampere Architecture, Nvidia A100 GPU

Extending the tensor Ops to FP64 is an interesting, if not surprising, design choice. Are there many applications sure to leverage this capability? Aside from HPL, of course.

I am suspecting that this is specifically targeted at the HPC market as the lack of FP64 has always been a hindrance to HPC deployment.

You have to remember that the HPC market is $35B today. HPE makes $3B a year alone from that, maybe more with Cray acquisition.

So it's no surprise that NVIDIA wants to position themselves on that market.

Plus, you have too look at the long game with MLX acquisition ( heavy player in the HPC market) and Cumulus. I wouldn't be surprised to see NVIDIA trying to bypass Intel/AMD completely and offer Direct to Interconnect device. Rather than the hybrid CPU + GPU box.

Is there a video or something?

The double-precision number probably best represents the generational improvement.

The 20x 32-bit floating point improvement is probably achieved by comparing doing full FP32 calculations on the previous generation vs doing TF32 calculations on Ampere. This would not be an apple-to-apple comparison as the TF32 result is less precise.

That said, it is probably not terribly important for deep learning at least, given the success of BF16.

I'm confused. Is there any relationship between the recent Ampere Arm64 servers (https://news.ycombinator.com/item?id=22475036) and Nvidia's "Ampere Architecture", or is it just a case of them using the same name?