Nvidia Hopper GPU Architecture and H100 Accelerator • Hacker News Board

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#11

Post by algo_trader » Tue, Mar 22, 2022, 5:25 PM UTC

From the nvidia page,

> 80 billion transistors

> Hopper H100 .. generational leap

> 9x at-scale training performance over A100

> 30x LLM inference throughput

> Transformer Engine .. speed .. 6x without losing accuracy

So another monster chip - same size of the Apple M1-max thingy ..

I guess it comes down to pricing. The A100 is already ridiculously expensive at $10K. They can this one at $50K and it would sell out?

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#12

Post by aninteger » Tue, Mar 22, 2022, 5:29 PM UTC

savant_penguin wrote: post #8

1 petaflop on a chip?? What is the catch?

Given that it's Nvidia, no Linux support. That's the catch.

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#13

Post by minimaxir » Tue, Mar 22, 2022, 5:31 PM UTC

So the product naming for Nvidia's server-GPUs by compute power now goes:

P100 -> V100 -> A100 -> H100

This is not confusing at all.

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#14

Post by komuher » Tue, Mar 22, 2022, 5:34 PM UTC

edf13 wrote: post #10

komuher wrote: post #7

1000 TFLOPS so i can run my GPT3 in under 100 ms locally :D If 1000 TFLOPS is possible to do in inference time then im speechless

At what costs I wonder?

I would assume about 30-40k usd but we'll see

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#15

Post by ScaleneTriangle » Tue, Mar 22, 2022, 5:35 PM UTC

aninteger wrote: post #12

savant_penguin wrote: post #8

1 petaflop on a chip?? What is the catch?

Given that it's Nvidia, no Linux support. That's the catch.

I thought that only applied to their consumer products.

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#16

Post by AHTERIX5000 » Tue, Mar 22, 2022, 5:36 PM UTC

aninteger wrote: post #12

savant_penguin wrote: post #8

1 petaflop on a chip?? What is the catch?

Given that it's Nvidia, no Linux support. That's the catch.

No Linux support? Guess I'll have to keep using Solaris with my A4000!

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#17

Post by gtirloni » Tue, Mar 22, 2022, 5:37 PM UTC

minimaxir wrote: post #13

So the product naming for Nvidia's server-GPUs by compute power now goes: P100 -> V100 -> A100 -> H100 This is not confusing at all.

Isn't it based on the architecture name?

https://en.wikipedia.org/wiki/Category:Nvidia_microarchitect...

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#18

Post by simulate-me » Tue, Mar 22, 2022, 5:37 PM UTC

aninteger wrote: post #12

savant_penguin wrote: post #8

1 petaflop on a chip?? What is the catch?

Given that it's Nvidia, no Linux support. That's the catch.

Nvidia provides Linux drivers for their server chips.

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#19

Post by kcb » Tue, Mar 22, 2022, 5:38 PM UTC

aninteger wrote: post #12

savant_penguin wrote: post #8

1 petaflop on a chip?? What is the catch?

Given that it's Nvidia, no Linux support. That's the catch.

That's a strange statement. The vast vast majority of these cards will be in systems running Linux.

Re: Nvidia Hopper GPU Architecture and H100 Accelerator

#20

Post by virtuallynathan » Tue, Mar 22, 2022, 5:39 PM UTC

How does a DGX Pod w/ the new 3.2Tbps per machine NVLINK switch compare to Tesla Dojo?