Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia will build 700-petaflop supercomputer for University of Florida

venturebeat.com

Post reply on HN
Print view View original

91–100 of 112 posts

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#91

Post by saganus »

lo_stronzo wrote: post #72
saganus wrote: post #60

Earlier quoted context omitted.

Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling? Is there anything useful someone could do with it or it's just too much of a problem to set it up and repurpose it?

> Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling? I cannot state with 100% clarity what happens after the systems are acquired by surplus vendors, but I can say that we have received certificates of what was "recycled". Once the surplus vendors take po…

Interesting! thanks for the insight.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#92

Post by etaioinshrdlu »

llukas wrote: post #7
etaioinshrdlu wrote: post #3

It seems like supercomputers are still generally many times more powerful than what AI researchers at top universities or orgs are using. Has any well known AI research been done on supercomputers? It seems like the case that literally just throwing money at the problem is a solid idea nowadays.

Cmon. Just look at requirements to train GPT-3 for example... https://lambdalabs.com/blog/demystifying-gpt-3/

I guess I had it in my head that they used something like 500 V100's -- which is still a far cry from a big supercomputer these days -- but they haven't published what hardware they used and for how long...

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#93

Post by boulos »

rrss wrote: post #84
boulos wrote: post #13

Sigh, I wish people wouldn’t say “peta flop ” for these. https://www.nvidia.com/en-us/data-center/a100/ is the most official reference. If you scroll to the bottom, you’ll see that an A100 part can do ~20 Teraflops (either FP32 or FP64 in little-matrix aka tensor mode). When they say “each A100 can do 5 petaflops”, they mean each DGX which has 8 such cards and thus they mean 600-ish something-ops per card. The genero…

You're the "disclaimer: I work for google cloud" person, right? Maybe you can help solve this problem (and it's a real one, it massively sucks that everyone counts whatever "flops" they want). Start here: https://cloud.google.com/tpu > Cloud TPU v3 Pod > 100+ petaflops Is the Cloud TPU v3 even functionally capable of computing in fp64 at any performance?

Yes, (and I considered including that disclaimer).

I give them shit about that all the time. I'm an equal-opportunity complainer.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#94

Post by fizixer »

boulos wrote: post #13

Sigh, I wish people wouldn’t say “peta flop ” for these. https://www.nvidia.com/en-us/data-center/a100/ is the most official reference. If you scroll to the bottom, you’ll see that an A100 part can do ~20 Teraflops (either FP32 or FP64 in little-matrix aka tensor mode). When they say “each A100 can do 5 petaflops”, they mean each DGX which has 8 such cards and thus they mean 600-ish something-ops per card. The genero…

I agree with you as well as with the response to your comment, in that both nVidia and Google are engaging in misleading claims about their TPUs. But I'd like to add two points:

- I don't think they can get away with it when it comes to Top-500 rankings. Those rankings are based on LINPACK scores and this supercomputer would end up not scoring high enough and will be placed in the right spot in the list. So it's not a big concern to me.

- "ML isn't scientific computing" goes both ways. Sure tensor-tera-ops are not teraflops, they're specific to operations involving artificial neural networks (ANNs). But when you take the claims from the past about how much computing power it'll take for rivaling that of a human brain, and folks came up with 100 petaflops, or 1 exaflop. Well those should not be called "f"lops either. Because when it comes to brain inspired computing, ANN tensor-tera-ops are a more reliable number than FP32 or FP64. And if it's 100 peta-ops of ANN compute, well we're already past that, and can easily create a supercomputer with 1 tensor-exa-ops. And that means we have reached the hardware capacity required to emulate human-level intelligence in a machine (i.e., the only thing missing is the right set of algorithms).

Conclusion: Tensor-tera-ops are not FP ops and should not be used for placement in the Top-500 list. But tensor-tera-ops have enough significance that it warrants creating a new list Top-500-Tensor and ranking Tensor-supercomputers on that list.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#95

Post by chansiky »

Blammar wrote: post #9
fluffything wrote: post #2

I'm a bit baffled. How high are the tuitions at the university of florida? This supercomputer is more powerful than many at the national labs and must cost a fortune (multiple 100 millions of dollars).

It's $50 million. That's all. See https://blogs.nvidia.com/blog/2020/07/21/university-of-flori... for the numbers. It's probable the hardware is being offered at a deep discount. The building and infrastructure will cost an additional $20m, covered by the University.

I'm glad they spent that $50 million on research rather than on next year's football program. Seems to be a decision most big state colleges would not make.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#96

Post by Todd »

boulos wrote: post #13

Sigh, I wish people wouldn’t say “peta flop ” for these. https://www.nvidia.com/en-us/data-center/a100/ is the most official reference. If you scroll to the bottom, you’ll see that an A100 part can do ~20 Teraflops (either FP32 or FP64 in little-matrix aka tensor mode). When they say “each A100 can do 5 petaflops”, they mean each DGX which has 8 such cards and thus they mean 600-ish something-ops per card. The genero…

Maybe we need a new unit of measure: the flip. Similar to how SN came up with the bi suffixes for binary (mebi vs. mega, etc.). The flip would be a small flop.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#97

Post by shmerl »

HeWhoLurksLate wrote: post #89
shmerl wrote: post #10

> It will also benefit from Nvidia’s suite of AI application frameworks Sounds like lock-in.

Is it lock-in when it's the only thing or the best thing on the market?

It is lock-in when it makes it very hard to move to anything else even if it's better. That's Nvidia's point. They don't play fair. They give this "for free" with hard strings attached to Nvidia.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#98

Post by [deleted] »

fizixer wrote: post #94
boulos wrote: post #13

Sigh, I wish people wouldn’t say “peta flop ” for these. https://www.nvidia.com/en-us/data-center/a100/ is the most official reference. If you scroll to the bottom, you’ll see that an A100 part can do ~20 Teraflops (either FP32 or FP64 in little-matrix aka tensor mode). When they say “each A100 can do 5 petaflops”, they mean each DGX which has 8 such cards and thus they mean 600-ish something-ops per card. The genero…

I agree with you as well as with the response to your comment, in that both nVidia and Google are engaging in misleading claims about their TPUs. But I'd like to add two points: - I don't think they can get away with it when it comes to Top-500 rankings. Those rankings are based on LINPACK scores and this supercomputer would end up not scoring high enough and will be placed in the right spot in the list. So it's not…

[deleted]

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#99

Post by [deleted] »

auslegung wrote: post #40
fluffything wrote: post #2

I'm a bit baffled. How high are the tuitions at the university of florida? This supercomputer is more powerful than many at the national labs and must cost a fortune (multiple 100 millions of dollars).

In addition to what everyone else has said, UF has over 50,000 students, making it one of the five largest universities in the USA, iirc.

[deleted]
Post reply on HN