It distresses me that we don’t have a language that can do a best effort parallelization of arbitrary loop like code across SIMD, multiple threads, multiple cores and GPU with a small directive. I don’t need it to be optimal, just … handy as an option! The last time I brought this up here, folks offered a bunch of options that don’t quite do this, and the best candidate was this 15 year old compiler project that is I…
It's called ParaSail. The next closest thing to ParaSail is ...
... literally just Rust.
Why? Because ParaSail has completely eliminated pointers, thereby preventing pointer aliasing. It can't be understated that pointer aliasing and alignment are the two biggest bottlenecks preventing autovectorization.
The people talking about better compilers, etc, just don't get it. It's not a compiler problem, it's a language semantics problem.
Pointer aliasing prevents parallelism full stop. If there is a single memory region and you perform a write to it, you have to assume that the write invalidates all data loaded from the pointers. If you make pointer aliasing illegal, then you have guaranteed that each pointer points to a distinct subset of the global memory, turning each pointer into a pointer to an isolated region. This means you have multiple regions you can write to in parallel. This is crucial, if you do not understand this you don't get parallel programming at all.
I mean think about it, this is the difference between having a four toilet bathroom with a single door or four doors.
The thing about alignment is not as easy to explain, but here is my attempt at it: If you allow the array to be misaligned at the front, then you have to run scalar code at the front. Same problem if you misalign at the end.
If you have a naked pointer and a for loop (think C), then the alignment problem alone precludes autovectorization without a complex scalar preamble. Autovectorization has to assume that the loop length could be anything, meaning it could be less than 8 elements to begin with. If the loop is always a multiple of 8 and always aligned, then the loop can be autovectorized even if the loop only does a single iteration.
Of course, after these critical blockages are gone you're still stuck with the problem of having to write branchless/non-diverging code.