Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia will build 700-petaflop supercomputer for University of Florida

venturebeat.com

Post reply on HN
Print view View original

41–50 of 112 posts

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#41

Post by abainbridge »

VadimPR wrote: post #32

How bad is this for the environment?

Seems like it has about 700/5 = 140 GPUs. And they are about 0.4kW each. Let's say it is 50% utilized for a year. That is 140 * 0.4 * 0.5 * 365 * 24 = 245280 kWh.

Let's say electricity generation creates 0.5 kg CO2e per kWh. So 122640 kg CO2e per year.

For comparison, driving a cars creates about 0.2kg CO2 per km. Or 0.32 kg per mile. So about the same as driving 122640 / 0.32 = 383250 miles per year. Call it 38 cars.

Edit: Corrected the above CO2 per km figure to fix a factor of 100 error.

Obviously there are a million things wrong with this analysis. For example, we don't know if they're going to use the machine to do climate modelling that will lead to a headline in the media that causes the green party to be elected ;-)

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#42

Post by anonymousDan »

pama wrote: post #38
abainbridge wrote: post #25

Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).

There is a trend towards larger models in the state of the art of deep learning research. The total cost of the on-chip memory on the NVidia GPU is still a good value proposition compared to current custom deep learning accelerators. This cost to get to large on-chip memory, combined with the flexibility of CUDA for other types of scientific applications, makes it harder for such accelerators to compete with Nvidia i…

Is this really true? At least for inference I've seen data showing the trend is rather for smaller models that exploit sparsity.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#43

Post by asutekku »

throwaway_pdp09 wrote: post #29
autokad wrote: post #26

Earlier quoted context omitted.

I'm in favor of getting high powered compute in everyone's hands, so this is great to me.

Why? I mean, we already have (my old 2013 desktop would - I guess - compete with a cray 1 from the 1980s). What do you expect will happen if people get more CPU?

The calculations we do nowaday are also completely different one did back in the 1980s. There’s still a lot you simply can not do on a home computer.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#44

Post by sillysaurusx »

Cthulhu_ wrote: post #33
abainbridge wrote: post #25

Does anyone know why everyone is still buying Nvidia instead of custom AI accelerators from other vendors? For example, on paper the new Graphcore machines look like an easy win, or at least a risk worth taking. (I see this particular supercomputer was funded by Nvidia but my question is about the general trend).

Risk mitigation, maintenance, resell value (?), support, reliability. The custom AI accelerators you mentioned, how long have those been in business for? How many units have they moved? How many generations of hardware have they produced? Will they still be around to replace or upgrade units in 5-10 years? How flexible are they in their workload? That's a lot of factors to keep in mind when you're spending millions o…

TPUs are perfectly positioned to capture this market. There are an endless number of reasons why, but to keep it short: Wanna see a magic trick? https://twitter.com/theshawwn/status/1286426454171975680

GPT-2 117M training at 1 million tokens/sec.

Now, I don't have experience with DGX clusters, so I'm not going to make a firm statement. What I will say is that I, as an outsider, managed to achieve a performance level that is ~unheard of for GPT-2 training. And you can too; TPUs are pervasive.

A TPUv2-512 isn't even as far as the gas pedal goes, either. v3-512 can train all of imagenet to 75.9% accuracy in 4 minutes: https://twitter.com/theshawwn/status/1223395022814339073

v3-1024 can do it in 2 minutes: https://twitter.com/theshawwn/status/1234654848114520065

I once attached a debugger to a training run during startup, after the infeed loop began (meaning it was feeding inputs to the TPU, but no training was happening yet; it was "winding up") and was shocked to discover that when I hit c to continue, it trained on all of imagenet in like 54 seconds. That blows the lid off of every perf result here (under "image classification"): https://mlperf.org/training-results-0-6

(It's not a fair comparison, but it was quite astonishing to see the raw horsepower in action.)

So, nVidia has some catching up to do. And I don't know if they'll be able to. The TPU ecosystem may be clunky at the moment, but boy is it effective. Your options are to invest your time in this ecosystem, which will likely be around in ten years, or in DGX-cluster-type knowledge, which ... might be less pervasive in 10 years.

The distinguishing feature of a TPU is that it has a CPU on board. In fact, it has a CPU with 300GB of memory for every 8 cores. Friggin' love these things.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#46

Post by lorenzhs »

eecc wrote: post #35
Cthulhu_ wrote: post #33

Earlier quoted context omitted.

Risk mitigation, maintenance, resell value (?), support, reliability. The custom AI accelerators you mentioned, how long have those been in business for? How many units have they moved? How many generations of hardware have they produced? Will they still be around to replace or upgrade units in 5-10 years? How flexible are they in their workload? That's a lot of factors to keep in mind when you're spending millions o…

Your supercomputer won’t last more than a couple years, 5 at max. Then it will simply be unjustifiably expensive to run when newer, more power efficient hardware becomes available. What matters is the software platform and compatibility: you don’t want to retool and port all your existing programs. That’s where having a platform - CUDA - becomes a deal-maker. What I don’t understand is how consumers - big, institutio…

Not sure why you're downvoted, HPC systems don't last more than ~6 years before they are decommissioned because they've become too power-hungry to justify their continued use. I just checked some of the systems I'm familiar with and they were all decommissioned after around 6 years of use (e.g. SuperMUC, 2012-2018; two smaller cluster at the university where I work: 08/2012-04/2017, 01/2014-03/2020).

I also attended a presentation at IPDPS 2018 by someone from one of the big US national labs where they talked about how absolutely huge of an undertaking it was to port their codes to make efficient use of GPUs. You don't just re-do all of that if the payoff isn't enormous.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#47

Post by lorenzhs »

abainbridge wrote: post #41
VadimPR wrote: post #32

How bad is this for the environment?

Seems like it has about 700/5 = 140 GPUs. And they are about 0.4kW each. Let's say it is 50% utilized for a year. That is 140 * 0.4 * 0.5 * 365 * 24 = 245280 kWh. Let's say electricity generation creates 0.5 kg CO2e per kWh. So 122640 kg CO2e per year. For comparison, driving a cars creates about 0.2kg CO2 per km. Or 0.32 kg per mile. So about the same as driving 122640 / 0.32 = 383250 miles per year. Call it 38 cars…

HPC system utilization is typically upwards of 90%, these things are not idle. There's almost always a queue of jobs waiting to run, at least in my experience.

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#48

Post by jcims »

abainbridge wrote: post #41
VadimPR wrote: post #32

How bad is this for the environment?

Seems like it has about 700/5 = 140 GPUs. And they are about 0.4kW each. Let's say it is 50% utilized for a year. That is 140 * 0.4 * 0.5 * 365 * 24 = 245280 kWh. Let's say electricity generation creates 0.5 kg CO2e per kWh. So 122640 kg CO2e per year. For comparison, driving a cars creates about 0.2kg CO2 per km. Or 0.32 kg per mile. So about the same as driving 122640 / 0.32 = 383250 miles per year. Call it 38 cars…

Cars generate closer to 122g of CO2 per km [1]. 122,000kg would be ~1 million km or ~50 cars.

1 - https://ec.europa.eu/clima/policies/transport/vehicles/cars_...

Re: Nvidia will build 700-petaflop supercomputer for University of Florida

#49

Post by abainbridge »

jcims wrote: post #48
abainbridge wrote: post #41

Earlier quoted context omitted.

Seems like it has about 700/5 = 140 GPUs. And they are about 0.4kW each. Let's say it is 50% utilized for a year. That is 140 * 0.4 * 0.5 * 365 * 24 = 245280 kWh. Let's say electricity generation creates 0.5 kg CO2e per kWh. So 122640 kg CO2e per year. For comparison, driving a cars creates about 0.2kg CO2 per km. Or 0.32 kg per mile. So about the same as driving 122640 / 0.32 = 383250 miles per year. Call it 38 cars…

Cars generate closer to 122g of CO2 per km [1]. 122,000kg would be ~1 million km or ~50 cars. 1 - https://ec.europa.eu/clima/policies/transport/vehicles/cars_...

Damn. I was confused by the fuel consumption being in per 100 km units. I've edited my post. Thanks.
Post reply on HN