My understanding is that once the Truffle/Graal language implementation framework lands with Java 9, you will be able to substitute method calls with platform-specific instructions. See page 62 of http://lafo.ssw.uni-linz.ac.at/papers/2015_CGO_Graal.pdf
Java and SIMD
21–30 of 40 posts
Re: Java and SIMD
#22Anyone knows how RyuJIT compares to Java8 and Java9 in a comparison like this?
ryujit does zero autovectorizing but
net has a nifty simd library thst lets you do it by hand. big plus is you can write once run on sse or avx or avx512. big downside is only a tiny fraction of simd instructions are supported, though they are working on improving that.
Re: Java and SIMD
#23Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
> From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning
Why do you think this is? Does C being difficult to reason about contribute (e.g. statefulness and not knowing what can modify what)?
I've done some SIMD and other optimisations on Android in C for graphics related algorithms and small changes and hints can make a x10 or more difference so I understand your point.
Re: Java and SIMD
#24Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. I…
Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?
For one, this is done on a core-by-core level. I believe Intel has the capability to monitor for this behaviour and do some dynamic throttling in certain models, which cloud vendors may have access to. I think they introduced some of this in Broadwell-E, where you can set the AVX offset per core, and the on-die PCU will throttle that core below the AVX base frequency, allowing the rest of the cores to remain at speed. These are typically controlled by BIOS, or MSRs.
We're currently trying to figure out how to do deal with this. Some ideas come from Google's CPI2 paper, and trying to dynamically schedule workloads with diversity if we think they interfere. Other thoughts have been simpler, like core pinning (knapsacking for latency, or throughput).
this is hard.
Disclaimer: These views represent my own, and not my employer's, or their vendor's views
Re: Java and SIMD
#25Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
This is why I like the SIMT model used in CUDA and OpenCL. Just give me a unified programming model for both SIMD vectors and multicore. Let me program each ALU as a separate thread. This requires CPU makers to wake up and give us branching intrinsics that work like in GPGPU: Just let the branched out ALUs sleep / do no-ops. As far as I understand Intel has done that with AVX-512 for KNL. This should be advertised much more. IMO one of the biggest mistakes with KNC was to only push OpenMP and neglect OpenCL. The promise of getting good performance by just recompiling OpenMP multicore code to KNC was snake oil - you have to rewrite all the vectorization in order to even get close to GPUs of the same generation, let alone GPUs released in subsequent years when Intel's production pipeline was stalled.
Re: Java and SIMD
#26Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
"Auto-vectorization is hard. Very hard. "
This varies very heavily depending on the programming language :)
C/C++ is not a language that easily enables one to guarantee things about aliasing or loop dependence.
Re: Java and SIMD
#27Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
> I would love to see smarter compilers, understanding the code, many steps over current hardwired pattern-matching based optimizations.
It'll be interesting when we get to that point. I wonder if it'll be in ML or compilers that "cognition" happens first?
Re: Java and SIMD
#28Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…
> From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning Why do you think this is? Does C being difficult to reason about contribute (e.g. statefulness and not knowing what can modify what)? I've done some SIMD and other optimisations on Android in C for graphics related algorithms and small changes and hints can make a x10…
The problem is usually determining when it's profitable, without any profiling, actually.
Additionally, C/C++ are languages where the default is "anything can alias anything" (restrict, until very recent standards, isn't the panacea people think it is) and worse, it's really easy to end up with loop dependences, as well as non-computable loop trip counts, as well. Don't forget alignment, too!
This means inserting runtime checks.
So, assuming the compiler can reorder the loop to vectorize it (and honestly, with polyhedral optimizations, it almost always can if it's at all possible to do so), the question is: is it worth it to vectorize a loop but have to insert 5-6 runtime checks to test for aliasing/etc.
The answer is usually no.
Vectorization, sadly, is not one of those things where more is always better.
Vectorizing every loop/straight line in a program will generally make things much much slower.
(because now you have limited the execution resources to do the computation :P)
Re: Java and SIMD
#29Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. I…
This doesn't use AVX512 though, and AVX512 is not supported in any mainstream JVM or CPU. It's only available in Xeon Phi, a HPC accelerator card with many slow cores.
Re: Java and SIMD
#30This is not a Java question. If a C compiler can do it then a Java Virtual Machine can do it, provided that the C code of the JVM makes it so. There are many implementations of the JVM so you really should be asking which JVMs do this. Oracle is not the only game in town, not to mention at least two open source JVMs where you could add whatever capabilities that you need.
On the other hand, if you are asking whether or not some magic compiler optimization will take your crappy code and make it run fast on SIMD, not only is that the wrong question but you have already lost the race.
The winners of the race asked the question, "How can we add a capability to our Java application to run computations fast using SIMD?" and they found number of ways to do this without relying on magic. It might be a bit of work to code because you have to do it with intent, like the old timers who placed code and data carefully on their drum memory computers to make the code run much faster. You can code with intent in any language on any platform, but because your intent is stronger than the asthetic perfection of the platform, things can look a little grungy to an outsider. Comment your code and document it well.
And ask yourself whether offloading the computation to a GPU might not be cheaper and even faster than SIMD.