What every developer should know about GPU computing

> [Foreach developer D, D] needs to know this.

> [Foreach developer D, D] doesn't need to know this. (No developers.)

But you could also have:

> [The logical conjuction of all developers[0]] needs to know this.

> [The logical conjuction of all developers] doesn't need to know this. (Not all developers.)

The former convention is clearly better unless I've missed something rather significant, but inherently broken features in a number of programming languages (eg perl 6 'junctive' operators) suggest the latter is well-established as a thing that exists.

0: ie, a linguistic fiction that has only those properties / needs only those things that are common to every developer.

Yes, you want to know as much as possible ( helps debugging/zooming in on issues since you don’t need to introduce an outsider to your problem, helps avoid errors), yes you need to specialize somewhere, no you can’t know everything and often don’t need to

But I put SIMD in the category of something that isn't necessary once one has learned other (more flexible) paradigms. I prefer MIMD and clusters/transputers, which seem to have died out by the 2000s. Today's status quo puts the onus on developers to move data manually, write shaders under arbitrary limitations on how many memory locations can be accessed simultaneously, duplicate their work with separate languages for GPU/CPU, know if various hardware is available for stuff like ray tracing, get locked into opinionated frameworks like OpenGL/Metal/Vulkan, etc etc etc. GPUs are on a side tangent that can never get me to where I want to go, so my experience over the last 25 years has been of a person living on the wrong timeline. I've commented about it extensively but it just feels like yelling into the void now.

Loosely, a scalable general purpose CPU working within the limitations of the end of Moore's law is multicore with local memories, sharing data through a copy-on-write content-addressable memory or other caching scheme which presents a single unified address space to allow the user to freely explore all methods of computation in a desktop computing setting. It uses standard assembly language but is usually programmed with something higher level like Erlang/Go, Octave/MATLAB or ideally a functional programming language like Julia. 3D rendering and AI libraries are written as a layer above that, they aren't fundamental.

It's interesting that GPUs have arrived at roughly the multicore configuration that I spoke of, but with drivers that separate the user from the bare-metal access needed to do general purpose MIMD. I had thought that FPGAs were the only way to topple GPU dominance, but maybe there is an opportunity here to write a driver that presents GPU hardware as MIMD with a unified memory. I don't know how well GPU cores handle integer math, but that could be approximated with the 32 bit int portion of a 64 bit float. Those sorts of tradeoffs may result in a MIMD machine running 10-100 times slower than a GPU, but still 10-100 times faster than a CPU. But scalable without the over-reliance on large caches and fast busses which stagnated CPUs since around 2007 when affordability and power efficiency took priority over performance due to the mobile market taking over. And MIMD machines can be clustered and form distributed compute networks like SETI@home with no changes to the code. To get a sense of how empowering that could be to the average user: it's like comparing BitTorrent to FTP, but for compute instead of data.