Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. I…
This doesn't use AVX512 though, and AVX512 is not supported in any mainstream JVM or CPU. It's only available in Xeon Phi, a HPC accelerator card with many slow cores.
Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
"Auto-vectorization is hard. Very hard. " This varies very heavily depending on the programming language :) C/C++ is not a language that easily enables one to guarantee things about aliasing or loop dependence.
but you can write it in a way that's conducive to better codegen, at the cost of developer sanity.
Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. I…
CPUs throttle down for AVX and AVX2 workloads as well. Hopefully your vectorization is sufficient to make up for the throttling.
This has got to be one of the best blog posts I've read in a while. It's clean, concise, has a clear use-case and is well benchmarked. Kudos to the author.
This is not a Java question. If a C compiler can do it then a Java Virtual Machine can do it, provided that the C code of the JVM makes it so. There are many implementations of the JVM so you really should be asking which JVMs do this. Oracle is not the only game in town, not to mention at least two open source JVMs where you could add whatever capabilities that you need. On the other hand, if you are asking whether…
Often it is not that simple to change JVM, especially if your application is very fine tuned for a specific GC. In such case, is it always worth to spend hundreds of work hours for migration (and testing it afterwards)? Other aspect is the technical support or other legal/contract bindings.
Besides, I wasn't asking for "magic compiler optimizations". I would prefer to use intrinsics directly. Is there a way to do that in Java?
Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
As most people mentioned, the issue in C/C++ is aliasing hence why the compiler cannot safely generate SIMD instructions. However, having worked in scientific computing for a few years on CFD solvers (written in C++), I have seen a very large advancement in what the compiler can do ( Intel compilers especially ). Our approach is to use OpenMP 4.0 directives as they also give you data scoping and alignment clauses. It's probably the best of both worlds where there is some much needed input from the programmer and then the compiler takes care of the rest. On our benchmark, a compute bound kernel would perform perhaps 10% better with hand tuned intrinsics vs the autovectorized version via OpenMP. And that is mostly because it removes some intermediary loads and stores when using short vectors in the autovectorized version which are usually done from a higher level cache anyway. In terms of portability, well, you can see why going for the OpenMP version is the safer bet. One important aspect to take into account is that for efficient autovectorization, you need to rewrite a lot of your kernels to use a vector programming paradigm (basically use short vectors of SIMD size -- which can be changed at compile time depending on uarch ) and trust me, then the compiler will be in its comfort zone. I've seen the compiler generate as good instructions for conditional branching as with hand tuned intrinsics ( AVX/AVX2 and AVX512 ). It can also do transposition from an AoS data structures to SoA in flight however our hand tuned kernels still outperform these since we know to use the lower latency instructions. From a performance perspective, without SIMD, we lose on average 2-3X in our simulation turn around time but getting our whole solver vectorized was a pretty mammoth task.
I've been wondering what the best way to implement SIMD in a JITless language was. While I'm a big proponent of static compilation (coding mostly in Go), this runtime inflexibility is a bit painful. Sure, you could compile multiple version, check the CPU and deliver the correct methods on runtime... but, that's not quite right.
Another issue is how you code it up - both in terms of delivering multiple versions depending on CPU support and in terms of shielding the developer from low-level assembly. I'm all for high level APIs (think `sum(a vec, b vec)`) that get compiled down to whatever is supported, but I haven't seen many good examples of this.
A few years back I decided to entertain myself by testing how smart today's smart compilers really are when it comes to auto-vectorization. I had this small and simple C application I'd written years earlier that tried to find inputs whose corresponding MD5 hashes started with certain bytes. It was a good base because it was obviously vectorizable. At first enabling the vectorizer didn't result in any changes to the…
I’ve had similar experiences. If I want vectorised code, I just write it myself using intrinsics or assembly. It’s fine if the compiler can autovectorise something I didn’t feel like doing by hand, but I’m not going to rely on heuristic voodoo to get the machine code I want for a hot loop. I wouldn’t mind a slightly nicer wrapper API for the intrinsics, though, something like glsl-sse2[1]. And that’s more or less wha…
"Auto-vectorization is hard. Very hard. " This varies very heavily depending on the programming language :) C/C++ is not a language that easily enables one to guarantee things about aliasing or loop dependence.
but you can write it in a way that's conducive to better codegen, at the cost of developer sanity.
Or you can design a language that is built all around it. Halide is pretty interesting, basically being a non-Turing complete DSL that lets you separate algorithm from scheduling:
This is not a Java question. If a C compiler can do it then a Java Virtual Machine can do it, provided that the C code of the JVM makes it so. There are many implementations of the JVM so you really should be asking which JVMs do this. Oracle is not the only game in town, not to mention at least two open source JVMs where you could add whatever capabilities that you need. On the other hand, if you are asking whether…
Often it is not that simple to change JVM, especially if your application is very fine tuned for a specific GC. In such case, is it always worth to spend hundreds of work hours for migration (and testing it afterwards)? Other aspect is the technical support or other legal/contract bindings. Besides, I wasn't asking for "magic compiler optimizations". I would prefer to use intrinsics directly. Is there a way to do tha…