Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

hpcwire.com

Post reply on HN
Print view View original

51–60 of 61 posts

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#51

Post by dekhn »

mortenjorck wrote: post #22

I'm curious what exactly distinguishes the "AI supercomputer" category from whatever the rest of the TOP500 are. Perlmutter's 120 petaflops peak would place it very favorably in TOP500's rankings, within the top five – if these flops aren't apples-to-oranges with what those rankings measure. Can anyone shed more light on the distinctions involved?

AI supercomputers report their performance in mixed-precision (bfloat16 or another format) FLOPs, and it's based on the peak performance of the machine, rather than realistic performance (like LINPACK). It's a terrible marketing approach that juices numbers in ways that are irrelevant to the existing supercomputer community.

The trend is more to call these things 'superpods' than 'supercomputers'.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#52

Post by dekhn »

crawdog wrote: post #37

Doing back of the napkin math you could accomplish this with 770 cloud instances in Google Cloud all running 8 attached GPUs. Now running lustre in the cloud is a pita so you would have to find out how to take advantage of cloud native storage. Running Rapids should be possible using Dataproc. Anyone have the budget to kick off the largest cloud native HPC workload?

google cloud does not have high performance networking. You want azure or maybe AWS if you wnt high performance networking.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#53

Post by dekhn »

prpl wrote: post #41
boulos wrote: post #39

Disclosure: I used to work on Google Cloud. Perlmutter seems like an awesome system. But, I think the “ai exaflops” is a “X GPUS times the NVIDIA peak rate”. The new sparsity features on A100 are promising, but haven’t been demonstrated to be nearly as awesome in practice (yet). It also all comes down to workloads: large-scale distributed training is a funny workload! It’s not like LINPACK. If you make your model com…

Perlmutter is a new kind of supercomputer - It’s mini-cloud disguised as a supercomputer smaller DOE projects to funnel their compute to. Cori was kind of that too, but IIRC kind of on accident because the Xeon Phi wasn’t available during delivery time, so the first “phase” was Haswell nodes. I’m not sure what this means for whole-machine reservation but hopefully that’s a thing of the past in any case. Ultimately th…

One of the reasons I left DOE for industry many years ago was that so much of the funding went to large supercomputers with about 8 users (the folks who had made their code scale to the largest size) while vast majority of people with compute needs went unsupported.

Tape is worthless. Tape is where you put data you never want to retrieve again.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#55

Post by chrisseaton »

Zenst wrote: post #47
chrisseaton wrote: post #4

Earlier quoted context omitted.

I don't think that's the sort of question a supercomputer like this can answer. It has additional compute power - it doesn't have any breakthrough in the AI itself.

True, but then how we define what AI is does have many answers more than it did back in Asimov's days. For me the ability to ask a question that is not clear to some and for others will have literary humour (Douglas Adams reference) would be telling about the AI. Some humans would pick up on that reference and the humour, some would be flummoxed and it is for me how intelligence answers questions that it does not ful…

> would be telling about the AI

Right… but there’s no point wondering because the answer is it can’t answer things like that.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#56

Post by l-lousy »

randomifcpfan wrote: post #21
1024core wrote: post #15

How does this compare with one of Google's TPU v4 pods? https://www.hpcwire.com/2021/05/20/google-launches-tpu-v4-ai... ?

From reading the two press releases, Perlmutter is 3.5 Exaflops vs Google TPUv4 at 1 Exaflop. Who knows how that compares in real life on real problems.

One pod is one exactions, and that release says they have dozens.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#57

Post by prpl »

dekhn wrote: post #53
prpl wrote: post #41

Earlier quoted context omitted.

Perlmutter is a new kind of supercomputer - It’s mini-cloud disguised as a supercomputer smaller DOE projects to funnel their compute to. Cori was kind of that too, but IIRC kind of on accident because the Xeon Phi wasn’t available during delivery time, so the first “phase” was Haswell nodes. I’m not sure what this means for whole-machine reservation but hopefully that’s a thing of the past in any case. Ultimately th…

One of the reasons I left DOE for industry many years ago was that so much of the funding went to large supercomputers with about 8 users (the folks who had made their code scale to the largest size) while vast majority of people with compute needs went unsupported. Tape is worthless. Tape is where you put data you never want to retrieve again.

I understand most of what you are saying, and it isn’t pleasant in many cases but I think the situation is improving.

Yes, all that happens, and most project computing (Office of Science) is usually pushed to NERSC, but it really depends on many factors. NERSC has a lot of users (much more than 8), probably the most out of the DOE complex, since it’s a user facility. Big projects (>$500M) don’t often have this problem and most of them will push for clusters they own at their home lab. Small projects are most likely to have trouble either in procuring resources or effectively using super computing resources.

A dirty secret is many users just use MPI for a job queue/coordination so their jobs “scale” when large reservations are required, but IB is not really exercised in those situations. When you realize that, it’s easy to realize most workflows scale (as long as disk is considered), but very annoying from a development perspective.

Tape isn’t worthless because you have to have a disaster plan or long term data preservation plan and it’s usually just a part of that, but some labs/projects have historically pushed as part of the integral data management plan for active/warm data and that is indeed worthless.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#58

Post by baruch »

airhead969 wrote: post #23

GPFS, Lustre, Panasas (PanFS / pNFS), BeeGFS are just about the only games in town for HPC pFSes. OrangeFS maybe. Also, I wouldn't want their bills for chilled water or electricity. My puny 96-thread EPYC ranges from 300 - 850 W and heats my room. :B

I work on the WekaFS (proprietary) that can definitely handle the relevant workloads and is in use in fairly large installations.

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#59

Post by scollet »

xiii1408 wrote: post #9
KETpXDDzR wrote: post #5

Now the important questions: 1) does it run FarCry? 2) are the GPUs limited in their ETH hashrate?

1) It runs Linux, so you're limited to 0 A.D. 2) No, it has the extremely expensive server GPUs, so their drivers should be normal.

I've run Far Cry 2 on Proton so the ceiling is open. My question is can we do Octree sims at scale???

Re: Berkeley Lab Debuts Perlmutter, World’s Fastest AI Supercomputer

#60

Post by dekhn »

prpl wrote: post #57
dekhn wrote: post #53

Earlier quoted context omitted.

One of the reasons I left DOE for industry many years ago was that so much of the funding went to large supercomputers with about 8 users (the folks who had made their code scale to the largest size) while vast majority of people with compute needs went unsupported. Tape is worthless. Tape is where you put data you never want to retrieve again.

I understand most of what you are saying, and it isn’t pleasant in many cases but I think the situation is improving. Yes, all that happens, and most project computing (Office of Science) is usually pushed to NERSC, but it really depends on many factors. NERSC has a lot of users (much more than 8), probably the most out of the DOE complex, since it’s a user facility. Big projects (>$500M) don’t often have this proble…

I really doubt people are running embarassingly parallel codes on NERSC supercomputers. I was a PI at NERSC and I was told explicitly, repeatedly, I couldn't do that. That's why I moved to Open Science Grid, and eventually Google's compute platform.

Nope, tape is just useless. Live but cold hard drives.

Post reply on HN