Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Think About GPUs

jax-ml.github.io

Post reply on HN
Print view View original

111–120 of 127 posts

Re: How to Think About GPUs

#111

Post by cwmoore »

HarHarVeryFunny wrote: post #102
radarsat1 wrote: post #54

Why haven't Nvidia developed a TPU yet?

Meaning what? Something less flexible? Less CUDA cores and more Tensor Cores? The majority of NVidia's profits (almost 90%) do come from data center, most of which is going to be neural net acceleration, and I'd have to assume that they have optimized their data center products to maximize performance for typical customer workloads. I'm sure that Microsoft would provide feedback to Nvidia if they felt changes were ne…

> most of which is going to be neural net acceleration

is it?

Re: How to Think About GPUs

#113

Post by shaklee3 »

corysama wrote: post #74
pklausler wrote: post #70

Earlier quoted context omitted.

So it's a "SIMD lane" that can itself perform actual SIMD instructions? I think you want a metaphor that doesn't also depend on its literal meaning.

Nvidia’s marketing team uses confusing terminology to make their product sound cooler than it is. An Intel “core” can perform AVX512 SIMD instructions that involve 16 lanes of 32-bit data. Intel cores are packaged in groups of up to 16. And, they use hyperthreading, speculative execution and shadow registers to cover latency. An Nvidia “Streaming Multiprocessor” can perform SIMD instructions on 32 lanes of 32-bits ea…

This is not true. GPUs are SIMT, but any given thread in those 32 in a warp can also issue SIMD instructions. see vector loads

Re: How to Think About GPUs

#114

Post by shaklee3 »

einpoklum wrote: post #91
pklausler wrote: post #70

Earlier quoted context omitted.

So it's a "SIMD lane" that can itself perform actual SIMD instructions? I think you want a metaphor that doesn't also depend on its literal meaning.

> that can itself perform actual SIMD instructions? Mostly, no; it can't really perform actual SIMD instructions itself. If you look at the SASS (the assembly language used on NVIDIA GPUs) I don't believe you'll see anything like that. In high-level code, you do have expressions involving "vectorized types", which look like they would translate into SIMD instruction, but they 'serialize', at the single thread level.…

not true; there are a lot of simd instructions on GPUs

Re: How to Think About GPUs

#115

Post by shaklee3 »

xtacy wrote: post #88
jacobaustin123 wrote: post #83

Earlier quoted context omitted.

I believe this is correct. For an H100, the 4 NVLink switches each have 64 ports supporting 25GB/s each, and each GPU uses a total of 18 ports. This gives us 450GB/s bandwidth within the node. But once you start trying to leave the node, you're limited by the per-node InfiniBand cabling, which only gives you 400GB/s out of the entire node (50GB / GPU).

Is it GBps (gigabytes per second) or Gbps (giga bits per second)? I see mixed usage in this comment thread so I’m left wondering what it actually is. The article is consistent and uses Gigabytes.

GBps

Re: How to Think About GPUs

#116

Post by physicsguy »

Q6T46nT668w6i3m wrote: post #50
physicsguy wrote: post #28

Earlier quoted context omitted.

It really isn't that hard to pivot. It's worth saying that if you were already writing OpenMP and MPI code then learning CUDA wasn't particularly difficult to get started, and learning to write more performant CUDA code would also help you write faster CPU bound code. It's an evolution of existing models of compute, not a revolution.

I agree that “learning CUDA wasn’t particularly difficult to get started,” there are Grand Canyon sized chasms between CUDA and its alternatives when attempting to crank performance.

Well, I think to a degree that depends what you're targeting.

Single socket 8 core CPU? Yes.

If you spent some time playing with trying to eke out performance on Xeon Phi and have done NUMA-aware code for multi socket boards and optimising for the memory hierarchy of L1/L2/L3 then it really isn't that different.

Re: How to Think About GPUs

#117

Post by dang »

almostgotcaught wrote: post #109
tverbeure wrote: post #105

Earlier quoted context omitted.

My takeaway was definitely not that all of Apple is using only one framework.

then please enlighten me: what does the sentence mean?

Can you please stop posting in the cross-examining, flamewar style? It's not what this site is for. We want people to learn from each other here.

https://news.ycombinator.com/newsguidelines.html

Re: How to Think About GPUs

#118

Post by einpoklum »

shaklee3 wrote: post #114
einpoklum wrote: post #91

Earlier quoted context omitted.

> that can itself perform actual SIMD instructions? Mostly, no; it can't really perform actual SIMD instructions itself. If you look at the SASS (the assembly language used on NVIDIA GPUs) I don't believe you'll see anything like that. In high-level code, you do have expressions involving "vectorized types", which look like they would translate into SIMD instruction, but they 'serialize', at the single thread level.…

not true; there are a lot of simd instructions on GPUs

Such as?

Re: How to Think About GPUs

#119

Post by einpoklum »

pklausler wrote: post #107
einpoklum wrote: post #91

Earlier quoted context omitted.

> that can itself perform actual SIMD instructions? Mostly, no; it can't really perform actual SIMD instructions itself. If you look at the SASS (the assembly language used on NVIDIA GPUs) I don't believe you'll see anything like that. In high-level code, you do have expressions involving "vectorized types", which look like they would translate into SIMD instruction, but they 'serialize', at the single thread level.…

Please see https://docs.nvidia.com/cuda/parallel-thread-execution/index... .

The "video instructions" are indeed another exception: Operations on sub-lanes of 32-bit values: 2x16 or 4x8. This is relevant for graphics/video work, where you often have Red, Green, Blue, Alpha channels of 8 bits each. Their use is uncommon (AFAICT) in CUDA compute work.
Post reply on HN