51–60 of 66 posts
This is a well written SIMD example I find these are the hardest topic to clearly explain because of how much is happening. Anyone got other nice links I've been doing a lot of vector processing so I need to learn!
Earlier quoted context omitted.
> My understanding (and I would be very happy for any clarifications/corrections!) is that you must use Metal Buffers (MTLBuffer) with the Apple Silicon GPU. If your data isn't already in a Metal Buffer (why would it be?), you have to do a copy into a buffer (but it will be extremely fast). You have to use Metal Buffers, but you don't have to copy, as long as it is properly page aligned [0]. The YUV data from both ca…
I have just been nerd sniped. EDIT - any hints on how to get a plain-old Swift array to be allocated in a conforming alignment (4096 bytes on Apple Silicon)?
When would you use SIMD vice a GPU? (Eg Vulkan comp shader) Is it easier to write for CPU, but you bring in the GPU shader if doing massively parallel ops vice just a few? I've skimmed a Rust lib that uses CPU SIMD (GLAM), and it used a diff syntax from normal, so I'm not positive it would be easier if you're familiar with the GPU process.
Earlier quoted context omitted.
Yeah I find the thing you generally want but which many languages don’t support well is to have a struct-of-arrays or data frame layout rather than an array of objects. Then you can do one of: Sort one array and rearrange other arrays the same way Compute the array of indices into another array such that array[index[i]] I think with simd you’d want to sort your array and simultaneously permute either a second array o…
Last I checked, Jai was going this direction (a type modifier that allowed one to do either AOS or SOA) but abandoned it (or at least, abandoned the particular syntax JB originally prototyped).
If you are exclusively moving numbers around in an array, SIMD sorting sounds great. But when you want to actually sort things, it's different. You have objects in an array, the objects will have one member which is a Key that you will sort the objects on. In order to create a sorted permutation of the list, you either need to rearrange the list of object pointers (fast), rearrange the memory of the objects (slow), o…
Yeah I find the thing you generally want but which many languages don’t support well is to have a struct-of-arrays or data frame layout rather than an array of objects. Then you can do one of: Sort one array and rearrange other arrays the same way Compute the array of indices into another array such that array[index[i]] I think with simd you’d want to sort your array and simultaneously permute either a second array o…
The ISPC User Guide was easy for me to work through, and there's a section on "Structure of Arrays" (SOA) programming [1].
This approach to vector-parallel programming was new to me, but got me to try some C code for the kernel transformation. My simple applications are probably fast enough in Python, but I thought it quite worthwhile to look at tiny kernels and the code the compiler generates.
- [0] https://ispc.github.io/index.html
- [1] https://ispc.github.io/ispc.html#structure-of-array-types
Earlier quoted context omitted.
It depends how much, where, and when you'd like the data to be sent? Most discrete GPUs cannot access the CPU memory directly, so you need to make a copy through the PCI bus, which can be slow. If you have small chunks of data (mining), or your destination is the screen (video game), it might make sense to use the GPU. If you need high-throughput, low latency, or your destination is something like a sound card (DAW),…
> Fast integrated GPUs like Apple's allow for directly accessing the main memory without copy, making the GPU more viable for general purposes. My understanding (and I would be very happy for any clarifications/corrections!) is that you must use Metal Buffers (MTLBuffer) with the Apple Silicon GPU. If your data isn't already in a Metal Buffer (why would it be?), you have to do a copy into a buffer (but it will be ext…
What sort of programs are you trying this with?
Has anyone been working on using SIMD on sorting networks? For the 16-input network, for each stage on the first half of the pipeline, all comparisons have no dependency and seems to be a good candidate for SIMD.
When would you use SIMD vice a GPU? (Eg Vulkan comp shader) Is it easier to write for CPU, but you bring in the GPU shader if doing massively parallel ops vice just a few? I've skimmed a Rust lib that uses CPU SIMD (GLAM), and it used a diff syntax from normal, so I'm not positive it would be easier if you're familiar with the GPU process.
Most cutting edge olap engines use SIMD extensively though how and where and how much is always left to imagination (confirmed with snowflake and databricks). At least practically speaking, I’ve sometimes amazed at how these engines are almost an order of magnitude faster for some computations when I have done similar computations using C and Java and assumed I knew the best possible performance for a typical cpu..
If you are exclusively moving numbers around in an array, SIMD sorting sounds great. But when you want to actually sort things, it's different. You have objects in an array, the objects will have one member which is a Key that you will sort the objects on. In order to create a sorted permutation of the list, you either need to rearrange the list of object pointers (fast), rearrange the memory of the objects (slow), o…
Yeah I find the thing you generally want but which many languages don’t support well is to have a struct-of-arrays or data frame layout rather than an array of objects. Then you can do one of: Sort one array and rearrange other arrays the same way Compute the array of indices into another array such that array[index[i]] I think with simd you’d want to sort your array and simultaneously permute either a second array o…
Earlier quoted context omitted.
Yeah I find the thing you generally want but which many languages don’t support well is to have a struct-of-arrays or data frame layout rather than an array of objects. Then you can do one of: Sort one array and rearrange other arrays the same way Compute the array of indices into another array such that array[index[i]] I think with simd you’d want to sort your array and simultaneously permute either a second array o…
Maybe Google's new "Rune" language will become prevalent https://github.com/google/rune , which supports SoA.