Live data from Hacker News

Implementing a GPU's programming model on a CPU

litherum.blogspot.com

31–40 of 61 posts

Re: Implementing a GPU's programming model on a CPU

#31
post #17

Earlier quoted context omitted.

[flagged]

Are you even reading the text you cite? That compares SIMD to SIMT. SIMD, not SPMD. SIMD is a more restrictive hardware paradigm, SPMD is just a mutiple copies of a single program running on multiple processes/threads, i.e., exactly what CUDA does. Most MPI-programs are SPMD, and really, the only conceptual difference between MPI and CUDA is that a thread's id is a point in a grid space that's relevant for scheduling…

Please don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse.

If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it. Note this one:

"Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that.""

Re: Implementing a GPU's programming model on a CPU

#32
post #7

This so-called GPU programming model has existed many decades before the appearance of the first GPUs, but at that time the compilers were not so good like the CUDA compilers, so the burden for a programmer was greater. As another poster has already mentioned, there exists a compiler for CPUs which has been inspired by CUDA and which has been available for many years: ISPC (Implicit SPMD Program Compiler), at https:/…

[flagged]

You've continued to post aggressively, attacking other users and generally posting in the flamewar style, even though we've asked you many times to stop. If this keeps up, we're going to have to ban you. I don't want to ban you, because you obviously have a lot of knowledge to share, but your aggressiveness destroys more than your knowledge contributes—a lot more, actually. Fortunately, there are plenty of other knowledgeable users here who treat their fellow users respectfully. Please be like them instead.

https://news.ycombinator.com/newsguidelines.html

Re: Implementing a GPU's programming model on a CPU

#35

> This is in contrast to SIMD, or "single instruction multiple data," where the programmer explicitly uses vector types and operations in their program. The SIMD approach is suited for when you have a single program that has to process a lot of data, whereas SIMT is suited for when you have many programs and each one operates on its own data This statement is comparing the SIMT model to SIMD. Can anyone explain the l…

SIMT let's a scheduler get clever about memory accesses, SIMD can practically only access memory linearly (scatter gather can do better but it's still usually quite linear) whereas SIMT can be much smarter in terms of having lots of similar bits of work going on in ways that use the bandwidth maximally and don't overlap.

Re: Implementing a GPU's programming model on a CPU

#36
post #35

> This is in contrast to SIMD, or "single instruction multiple data," where the programmer explicitly uses vector types and operations in their program. The SIMD approach is suited for when you have a single program that has to process a lot of data, whereas SIMT is suited for when you have many programs and each one operates on its own data This statement is comparing the SIMT model to SIMD. Can anyone explain the l…

SIMT let's a scheduler get clever about memory accesses, SIMD can practically only access memory linearly (scatter gather can do better but it's still usually quite linear) whereas SIMT can be much smarter in terms of having lots of similar bits of work going on in ways that use the bandwidth maximally and don't overlap.

https://developer.nvidia.com/blog/how-access-global-memory-e...

SIMT still expects coalesced memory access that's close together otherwise performance falls off a cliff

Re: Implementing a GPU's programming model on a CPU

#37

In addition to ISPC, some of this is also done in software fallback implementations of GPU APIs. In the open source world we have SwiftShader and Lavapipe, and on Windows we have WARP[1]. It's sad to me that Larrabee didn't catch on, as that might have been a path to a good parallel computer, one that has efficient parallel throughput like a GPU, but also agility more like a CPU, so you don't need to batch things int…

The recent "AI chip" proposals (Tenstorrent, Esperanto Technologies etc.) seem to be quite similar to the old Larrabee design, except based on RISC-V as opposed to x86. So we might see that happen after all.

Re: Implementing a GPU's programming model on a CPU

#38
post #24

Earlier quoted context omitted.

Ok dude, give me an actual argument. Tell me a substantial way that SIMT is different from SPMD

Subgroup operations[1]. In SIMT, they are fairly easily modeled[2] as communication between the different threads running on the same SIMD, and in most cases explicitly expose the predication mask. In fact, subgroupBallot(true) is a common idiom to extract that mask, and also to determine the subgroup size if run in subgroup uniform control flow. Generally they have good performance characteristics. In SPMD, subgroup…

This is true to some extent, although certainly SPMD has subgroups and reduction operations on those are fairly common.

And you are essentially repeating my point from before, that the fact that an operation among the SIMT threads can be scheduled as one SIMD op is the only special behavior. My example was regular vectorized SIMD ops, but your example of reductions are essentially the same point. Shuffles and horizontal adds are present in SIMT just like they are in regular SIMD.

These same reductions exist in contexts where the model would traditionally be called SPMD. But because the concepts map onto a different level of the system we might not easily recognize them as the same. The same kind of subgrouping happens in a multinode SPMD context in terms of NUMA regions and nodes, and an MPI implementation can handle reductions on a within-node or within-NUMA-region group of ranks more efficiently than a group containing ranks assigned to multiple meaningful hardware regions.

So I agree with you that the one thing SIMT has to differentiate it from SPMD is the SIMD-like scheduling of multiple threads -- and this was exactly my point from before -- but even here we find that SPMD has the same concepts, just not as explicitly, because the stack typically used in SPMD is more flexible, and SIMT can take advantage of specialized hardware ops.

Optimizations like this do not necessarily mean that the model isn't SPMD though. It's still a single program (a kernel) processing multiple data (indexed by the thread id). The one key idea of SIMT is really all about the batch scheduling of threads, the fact that multiple threads from a warp can be executed on one SM partition (using CUDA terms here) using a single instruction and a mask.

SIMT emphasizes the GPU scheduler, but it still fits completely within SPMD. All SIMT technologies are SPMD technologies, but not all SPMD technologies are SIMT technologies. I'm not convinced there is enough new in SIMT to warrant the branding using a new architecture type in the taxonomy. They could have just as well emphasized the thread scheduling as an optimization in an SPMD context.

Re: Implementing a GPU's programming model on a CPU

#39
post #31
post #17

Earlier quoted context omitted.

Are you even reading the text you cite? That compares SIMD to SIMT. SIMD, not SPMD. SIMD is a more restrictive hardware paradigm, SPMD is just a mutiple copies of a single program running on multiple processes/threads, i.e., exactly what CUDA does. Most MPI-programs are SPMD, and really, the only conceptual difference between MPI and CUDA is that a thread's id is a point in a grid space that's relevant for scheduling…

Please don't respond to a bad comment by breaking the site guidelines yourself. That only makes things worse. If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it. Note this one: " Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that." "

My bad, I'll do better.

Re: Implementing a GPU's programming model on a CPU

#40

In addition to ISPC, some of this is also done in software fallback implementations of GPU APIs. In the open source world we have SwiftShader and Lavapipe, and on Windows we have WARP[1]. It's sad to me that Larrabee didn't catch on, as that might have been a path to a good parallel computer, one that has efficient parallel throughput like a GPU, but also agility more like a CPU, so you don't need to batch things int…

The recent "AI chip" proposals (Tenstorrent, Esperanto Technologies etc.) seem to be quite similar to the old Larrabee design, except based on RISC-V as opposed to x86. So we might see that happen after all.

Yes, I've got my eye on those and am hopeful. Do you know of any meaty technical description of the programming model? All I've been able to find so far is fairly high level marketing material. At least for Tenstorrent Jim Keller has promised that the software stack will be open sourced, something I'm looking forward to.
Post reply on HN