Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Implementing a GPU's programming model on a CPU

litherum.blogspot.com

Post reply on HN
Print view View original

11–20 of 61 posts

Re: Implementing a GPU's programming model on a CPU

#11

Post by mattpharr »

adrian_b wrote: post #7

This so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https:/…

Also related as far as the programming model: C* on Connection Machines and the C dialect used on MasPar systems in the 1980s.

There is some discussion in the ispc paper: https://pharr.org/matt/assets/ispc.pdf

Re: Implementing a GPU's programming model on a CPU

#14

Post by mathisfun123 »

adrian_b wrote: post #7

This so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https:/…

[flagged]

Re: Implementing a GPU's programming model on a CPU

#15

Post by raphlinus »

In addition to ISPC, some of this is also done in software fallback implementations of GPU APIs. In the open source world we have SwiftShader and Lavapipe, and on Windows we have WARP[1].

It's sad to me that Larrabee didn't catch on, as that might have been a path to a good parallel computer, one that has efficient parallel throughput like a GPU, but also agility more like a CPU, so you don't need to batch things into huge dispatches and wait RPC-like latencies for them to complete. Apparently the main thing that sunk it was power consumption.

[1]: https://learn.microsoft.com/en-us/windows/win32/direct3darti...

Re: Implementing a GPU's programming model on a CPU

#16

Post by samsartor »

I helped make a really cursed RISC-V version of this for a class project last year! The idea was to first compile each program to WASM using clang, and lower the WASM back to C but this time with all opcodes implemented in terms of the RISC-V vector intrinsics. That was a hack to be sure, but a surprisingly elegant one since 1. WASM's structured control flow maps really well to lane masking 2. Stack and local values easily use "structure of arrays" layout 3. Heap values easily use "array of structures" layout

It never went anywhere but the code is still online if anyone wants to stare directly at the madness: https://gitlab.com/samsartor/wasm2simt

Re: Implementing a GPU's programming model on a CPU

#17

Post by tovej »

mathisfun123 wrote: post #14
adrian_b wrote: post #7

This so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https:/…

[flagged]

Are you even reading the text you cite? That compares SIMD to SIMT. SIMD, not SPMD. SIMD is a more restrictive hardware paradigm, SPMD is just a mutiple copies of a single program running on multiple processes/threads, i.e., exactly what CUDA does.

Most MPI-programs are SPMD, and really, the only conceptual difference between MPI and CUDA is that a thread's id is a point in a grid space that's relevant for scheduling and memory locality. Oh wait, that's true for MPI as well, only the scheduling and memory locality relates to CPU cores and nodes, instead of CUDAs SMPs and blocks.

The only real justification for a different programming model is that SIMT creates opportunities for implicit SIMD on a thread scheduling level if you've written your program in a way that the warps don't diverge.

Re: Implementing a GPU's programming model on a CPU

#18

Post by JonChesterfield »

adrian_b wrote: post #7

This so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https:/…

SIMT and SIMD are different things. It's fortunate that they have different names.

A GPU is a single instruction multiple data machine. That's what the predicated vector operations are. 32 floats at a time, each with a disable bit.

Cuda is a single instruction multiple thread language. You write code in terms of one float and branching on booleans, as if it was a CPU, with some awkward intrinsics for accessing the vector units in the GPU.

That is, the programming model of a GPU ISA and that of Cuda are not the same. The GPU gives you vector instructions. Cuda gives you (mostly) scalar instructions and a compiler that deals with this mismatch, lowering branches to changes in exec mask and so forth.

With my numerical library hat on, I hate this. Programming a simd machine through a simt language means trying to get the compiler to transform the control flow into the thing you could easily write using vector instructions.

With my compiler implementer hat on, I hate this. It gives you two control flow graphs intertwined and a really bad time in register allocation.

It's not totally clear to me why simt won out over writing the vector operations. I'm certainly in the minority opinion here.

Re: Implementing a GPU's programming model on a CPU

#19

Post by mathisfun123 »

tovej wrote: post #17
mathisfun123 wrote: post #14

Earlier quoted context omitted.

[flagged]

Are you even reading the text you cite? That compares SIMD to SIMT. SIMD, not SPMD. SIMD is a more restrictive hardware paradigm, SPMD is just a mutiple copies of a single program running on multiple processes/threads, i.e., exactly what CUDA does. Most MPI-programs are SPMD, and really, the only conceptual difference between MPI and CUDA is that a thread's id is a point in a grid space that's relevant for scheduling…

> Are you even reading the text you cite?

Yes I have read it enough times that I knew exactly where it was in the book.

> That compares SIMD to SIMT.

Yes and so does the op I responded to.

> SIMD, not SPMD. SIMD is a more restrictive hardware paradigm,

Thanks but I was already aware. The op was the one that conflated SPMD and SIMD, not me.

> SPMD is just a mutiple copies of a single program running on multiple processes/threads, i.e., exactly what CUDA does.

CUDA is a C language extension that enables you to write host and device code in the same source. That's it. SIMT is the model of compute on NVIDIA SMs.

> The only real justification for a different programming model is that SIMT creates opportunities for implicit SIMD on a thread scheduling level if you've written your program in a way that the warps don't diverge.

No that's incorrect. SIMT is implemented using predicated execution and instruction replay exactly so that warps can diverge.

Again, it's amazing you guys are so sure that NVIDIA pulled a fast one on just the whole community/industry and didn't actually innovate but it turns out you're actually missing some pieces of the puzzle.

Re: Implementing a GPU's programming model on a CPU

#20

Post by hasmanean »

adrian_b wrote: post #7

This so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https:/…

Subtle difference. A parallel_for can have asynchronous threads. They can all divergen and run independent instructions (due to if statements etc)

SIMT means multiple processors all executing the exact same instruction. One instruction decoder, but like 64 execution pipelines.

Post reply on HN