[deleted]
Nvidia will build 700-petaflop supercomputer for University of Florida
21–30 of 112 posts
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#22I'm a bit baffled. How high are the tuitions at the university of florida? This supercomputer is more powerful than many at the national labs and must cost a fortune (multiple 100 millions of dollars).
No, those national lab supercomputers typically quote double-precision flops, whereas this appears to be about float16 tensor ops for inner product (they call it "AI flops"). The A100 is 9.7 general purpose Tflops of float64, versus 624 Tflops of float16 / bfloat16 (perhaps also just tensor cores not general purpose)? So quoting it the same way, this Florida machine would only be a (700 * (9.7 / 624)) = 10.9 Pflop su…
The machine will be 140 nodes, which is half the size of the current #7 on the top500 list which is otherwise identical: https://top500.org/system/179842/
That #7 machine got 27.6 Pflops on HPL, so I would guess the Florida machine will get roughly half of that.
(Please keep in mind the tensor cores on the A100 can do FP64, so the theoretical peak flops per GPU is 19.5)
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#23I'm a bit baffled. How high are the tuitions at the university of florida? This supercomputer is more powerful than many at the national labs and must cost a fortune (multiple 100 millions of dollars).
It's $50 million. That's all. See https://blogs.nvidia.com/blog/2020/07/21/university-of-flori... for the numbers. It's probable the hardware is being offered at a deep discount. The building and infrastructure will cost an additional $20m, covered by the University.
https://lambdalabs.com/blog/demystifying-gpt-3/
According to ^, that's about 11 GPT-3s trainings worth in the cloud.
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#24This being NVIDIA, University of Florida would be wise to hire an independent company to benchmark their "700-petaflop".
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#25Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#26I'm a bit baffled. How high are the tuitions at the university of florida? This supercomputer is more powerful than many at the national labs and must cost a fortune (multiple 100 millions of dollars).
It's $50 million. That's all. See https://blogs.nvidia.com/blog/2020/07/21/university-of-flori... for the numbers. It's probable the hardware is being offered at a deep discount. The building and infrastructure will cost an additional $20m, covered by the University.
I'm in favor of getting high powered compute in everyone's hands, so this is great to me.
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#27Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).
Do those custom AI accelerators work with PyTorch and Tensorflow? The graphcore website says that support with PyTorch aten is available in 2020, but it's not -- and Aten is inference only, no autograd, and no extensions. Without those, we can't train on it.
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#28Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).
Do those custom AI accelerators work with PyTorch and Tensorflow? The graphcore website says that support with PyTorch aten is available in 2020, but it's not -- and Aten is inference only, no autograd, and no extensions. Without those, we can't train on it.
Graphcore claims they do. I don't know true that is though.
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#29Earlier quoted context omitted.
It's $50 million. That's all. See https://blogs.nvidia.com/blog/2020/07/21/university-of-flori... for the numbers. It's probable the hardware is being offered at a deep discount. The building and infrastructure will cost an additional $20m, covered by the University.
I'm in favor of getting high powered compute in everyone's hands, so this is great to me.
Why?
I mean, we already have (my old 2013 desktop would - I guess - compete with a cray 1 from the 1980s). What do you expect will happen if people get more CPU?
Re: Nvidia will build 700-petaflop supercomputer for University of Florida
#30Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).
Many people use HPC for things that are not AI/ML, such as CFD. Those applications are typically written using MPI/CUDA