Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia will build 700-petaflop supercomputer for University of Florida

venturebeat.com

Post reply on HN
Print view View original

51–60 of 112 posts

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#51

Post by eecc »

dannyw wrote: post #36
eecc wrote: post #35

Earlier quoted context omitted.

Your supercomputer won’t last more than a couple years, 5 at max. Then it will simply be unjustifiably expensive to run when newer, more power efficient hardware becomes available. What matters is the software platform and compatibility: you don’t want to retool and port all your existing programs. That’s where having a platform - CUDA - becomes a deal-maker. What I don’t understand is how consumers - big, institutio…

CUDA is valuable and seen as a positive, even if it's vendor locked. It's battle tested. Engineers are familiar with it. Support is provided because it's all under one roof. Performance targets, roadmap timeframes, etc can be assured. The same reason a big institutional consumer will happily pay Microsoft for Windows Enterprise, instead of demanding Microsoft rebuild Windows using open source technologies. A supercom…

I’m not saying CUDA is not valuable. Heck, proprietary has a ton of value... see how Intel owes much of its position to its compiler (the FORTRAN one in particular) suite.

Of course you don’t get to dominate a field to such an extent without some redeeming positive quality.

What I’m arguing - or more appropriately, what I’m wondering - is why on Earth aren’t consumer consortiums not insisting on open-standard platforms.

I get how you want to eek every drop of performance out of your tools without making it your only mission (that would be the research you’re using the tool for), but tying yourself to a single proprietary vendor defeats that purpose imho

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#52

Post by jcims »

abainbridge wrote: post #49
jcims wrote: post #48

Earlier quoted context omitted.

Cars generate closer to 122g of CO2 per km [1]. 122,000kg would be ~1 million km or ~50 cars. 1 - https://ec.europa.eu/clima/policies/transport/vehicles/cars_...

Damn. I was confused by the fuel consumption being in per 100 km units. I've edited my post. Thanks.

Yeah who isn't hahaha

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#54

Post by jabl »

abainbridge wrote: post #41
VadimPR wrote: post #32

How bad is this for the environment?

Seems like it has about 700/5 = 140 GPUs. And they are about 0.4kW each. Let's say it is 50% utilized for a year. That is 140 * 0.4 * 0.5 * 365 * 24 = 245280 kWh. Let's say electricity generation creates 0.5 kg CO2e per kWh. So 122640 kg CO2e per year. For comparison, driving a cars creates about 0.2kg CO2 per km. Or 0.32 kg per mile. So about the same as driving 122640 / 0.32 = 383250 miles per year. Call it 38 cars…

It has 140 nodes, each with 8 GPU's. Add in extra infra like storage, networking etc. and you're looking at around 700 kW at 100% utilization.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#55

Post by fluffy87 »

abainbridge wrote: post #25

Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).

I reimplemented one benchmark using TF and the speed up wasn’t the 10x or so hey we’re claiming, but more like 1x..

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#56

Post by saberience »

sillysaurusx wrote: post #44
Cthulhu_ wrote: post #33

Earlier quoted context omitted.

Risk mitigation, maintenance, resell value (?), support, reliability. The custom AI accelerators you mentioned, how long have those been in business for? How many units have they moved? How many generations of hardware have they produced? Will they still be around to replace or upgrade units in 5-10 years? How flexible are they in their workload? That's a lot of factors to keep in mind when you're spending millions o…

TPUs are perfectly positioned to capture this market. There are an endless number of reasons why, but to keep it short: Wanna see a magic trick? https://twitter.com/theshawwn/status/1286426454171975680 GPT-2 117M training at 1 million tokens/sec. Now, I don't have experience with DGX clusters, so I'm not going to make a firm statement. What I will say is that I, as an outsider, managed to achieve a performance level…

1 milion per second of anything isn't very fast. Unless you're running that on a mobile phone.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#57

Post by wil421 »

auslegung wrote: post #40
fluffything wrote: post #2

I'm a bit baffled. How high are the tuitions at the university of florida? This supercomputer is more powerful than many at the national labs and must cost a fortune (multiple 100 millions of dollars).

In addition to what everyone else has said, UF has over 50,000 students, making it one of the five largest universities in the USA, iirc.

Correct. Florida has 3 of the top 10 by enrollment. UCF, UF, FIU, and USF. Georgia made the list, go dawgs.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#58

Post by abainbridge »

jabl wrote: post #54
abainbridge wrote: post #41

Earlier quoted context omitted.

Seems like it has about 700/5 = 140 GPUs. And they are about 0.4kW each. Let's say it is 50% utilized for a year. That is 140 * 0.4 * 0.5 * 365 * 24 = 245280 kWh. Let's say electricity generation creates 0.5 kg CO2e per kWh. So 122640 kg CO2e per year. For comparison, driving a cars creates about 0.2kg CO2 per km. Or 0.32 kg per mile. So about the same as driving 122640 / 0.32 = 383250 miles per year. Call it 38 cars…

It has 140 nodes, each with 8 GPU's. Add in extra infra like storage, networking etc. and you're looking at around 700 kW at 100% utilization.

You're right. The article is wrong. It said:

> A single A100 GPU’s 54 billion transistors can execute 5 petaflops of performance, according to Nvidia.

It is a single system of 8 GPUs that can do 5 petaflops.

I can't edit my post anymore. Multiply my numbers by 8, or a bit more if you want to take into account higher utilization and the other overheads you mention. So ~500 cars. This is my final offer.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#59

Post by throwaway_pdp09 »

eecc wrote: post #37
throwaway_pdp09 wrote: post #29

Earlier quoted context omitted.

Why? I mean, we already have (my old 2013 desktop would - I guess - compete with a cray 1 from the 1980s). What do you expect will happen if people get more CPU?

Better simulations? Successfully working on previously intractable problems?

But most people ie. not 'everyone' as the post talked about, is doing that. Most just want web browsing + LibreOffice stuff. Oh, and games I suppose.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#60

Post by saganus »

lorenzhs wrote: post #46
eecc wrote: post #35

Earlier quoted context omitted.

Your supercomputer won’t last more than a couple years, 5 at max. Then it will simply be unjustifiably expensive to run when newer, more power efficient hardware becomes available. What matters is the software platform and compatibility: you don’t want to retool and port all your existing programs. That’s where having a platform - CUDA - becomes a deal-maker. What I don’t understand is how consumers - big, institutio…

Not sure why you're downvoted, HPC systems don't last more than ~6 years before they are decommissioned because they've become too power-hungry to justify their continued use. I just checked some of the systems I'm familiar with and they were all decommissioned after around 6 years of use (e.g. SuperMUC, 2012-2018; two smaller cluster at the university where I work: 08/2012-04/2017, 01/2014-03/2020). I also attended…

Out of curiosity, do you know what happens to the decomissioned hardware? is it scrapped for useful parts like perhaps the PSUs? or is gold and other metals extracted from the chips and the metal sold for recycling?

Is there anything useful someone could do with it or it's just too much of a problem to set it up and repurpose it?

Post reply on HN