Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Java and SIMD

prestodb.rocks

Post reply on HN
Print view View original

11–20 of 40 posts

Re: Java and SIMD

#11

Post by YSFEJ4SWJUVU6 »

faragon wrote: post #6

Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…

A few years back I decided to entertain myself by testing how smart today's smart compilers really are when it comes to auto-vectorization.

I had this small and simple C application I'd written years earlier that tried to find inputs whose corresponding MD5 hashes started with certain bytes. It was a good base because it was obviously vectorizable.

At first enabling the vectorizer didn't result in any changes to the binaries. I then correctly guessed that (potentially) calling printf function inside a hot loop might confuse it. After slight refactoring I got the compiler to output SSE instructions, which resulted in a nice 2.5× testing speed over the original (incidentally even without auto-vectorizing the refactored code resulted in faster binaries, which is not all that surprising).

Anyway, I also rewrote the application to use intrinsics. I hadn't used them before myself, but it didn't really take much time at all familiarize myself with them and write the code, and it was indeed quite a bit faster than what the compiler was capable of with resulting binary having 14× speed compared to the original, or over 5× compared to what the compiler could achieve without explicit hints from intrinsics.

Edit: added back a few words I had accidentally removed when rearranging sentences, causing a confusing incomplete sentence. Corrected comparing figures like for like.

Re: Java and SIMD

#12

Post by faragon »

YSFEJ4SWJUVU6 wrote: post #11
faragon wrote: post #6

Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…

A few years back I decided to entertain myself by testing how smart today's smart compilers really are when it comes to auto-vectorization. I had this small and simple C application I'd written years earlier that tried to find inputs whose corresponding MD5 hashes started with certain bytes. It was a good base because it was obviously vectorizable. At first enabling the vectorizer didn't result in any changes to the…

That matches my experience. With auto-vectorization you get some speed-up helping the compiler (not always obvious, often requiring +1 increments, etc.), but for full speed you need to do handwritten SIMD intrinsics. I would like to have at least 50% of the optimal by the compiler, without intrinsics (and using intrinsics for the most critical code).

Re: Java and SIMD

#13

Post by foota »

YSFEJ4SWJUVU6 wrote: post #11
faragon wrote: post #6

Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…

A few years back I decided to entertain myself by testing how smart today's smart compilers really are when it comes to auto-vectorization. I had this small and simple C application I'd written years earlier that tried to find inputs whose corresponding MD5 hashes started with certain bytes. It was a good base because it was obviously vectorizable. At first enabling the vectorizer didn't result in any changes to the…

Would be interesting to see the difference in the compiled assembly.

Re: Java and SIMD

#14

Post by [deleted] »

YSFEJ4SWJUVU6 wrote: post #11
faragon wrote: post #6

Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…

A few years back I decided to entertain myself by testing how smart today's smart compilers really are when it comes to auto-vectorization. I had this small and simple C application I'd written years earlier that tried to find inputs whose corresponding MD5 hashes started with certain bytes. It was a good base because it was obviously vectorizable. At first enabling the vectorizer didn't result in any changes to the…

[deleted]

Re: Java and SIMD

#15

Post by briane80 »

aardvark179 wrote: post #4

There's room for two approaches in Java really. The JIT can be smart and use SIMD instructions where it can see they are applicable, but there's also room for a small DSL like API that allows library authors and other very experienced users to express a suitable algorithm and have it easily translated into the SIMD instructions available at runtime. Anybody interested in the latter should take a look at the work bein…

Annotation based compiler hints?

Re: Java and SIMD

#16

Post by evincarofautumn »

YSFEJ4SWJUVU6 wrote: post #11
faragon wrote: post #6

Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…

A few years back I decided to entertain myself by testing how smart today's smart compilers really are when it comes to auto-vectorization. I had this small and simple C application I'd written years earlier that tried to find inputs whose corresponding MD5 hashes started with certain bytes. It was a good base because it was obviously vectorizable. At first enabling the vectorizer didn't result in any changes to the…

I’ve had similar experiences. If I want vectorised code, I just write it myself using intrinsics or assembly. It’s fine if the compiler can autovectorise something I didn’t feel like doing by hand, but I’m not going to rely on heuristic voodoo to get the machine code I want for a hot loop. I wouldn’t mind a slightly nicer wrapper API for the intrinsics, though, something like glsl-sse2[1].

And that’s more or less what I’m planning to do in a programming language I’m working on, actually—if you use a SIMD-compatible array type, the compiler will try to keep it in a vector register, and some operations will be faster (e.g., “+” on two Float32^4 values will compile to an addps) but it’s up to the programmer to use the instructions they actually want, or tell the compiler with a macro “please vectorise this loop or warn me about why you can’t”.

[1] https://github.com/LiraNuna/glsl-sse2

Re: Java and SIMD

#17

Post by stusmall »

marmaduke wrote: post #7

Why not use some OpenCL in some form either raw or Aparapi?

Crossing the JNI barrier is expensive performance-wise. Unless it's a longer running, CPU heavy calculation its usually best to stay in pure Java. There are some things that going out to another high performance framework will speed things up, but the bar is a bit higher because of the JNI overhead.

Re: Java and SIMD

#18

Post by desdiv »

stu2010 wrote: post #10
sargun wrote: post #9

Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. I…

Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?

They don't? AFAIK no cloud vendor guarantees consistent performance on their multi-tenancy offerings (at least none backed by SLAs). Vendors sell both dedicated tenancy, with consistent performance, at a higher price and shared tenancy, with variable performance, at a lower price and expect customers who choose the latter to deal with the variance on their own.

Large companies run a benchmark after each instance startup to check how noisy that instance's neighbors are. If it's bad then they simply terminate the instance and start another one[0].

[0] https://www.reddit.com/r/aws/comments/547xbx/netflix_found_5...

Re: Java and SIMD

#19

Post by [deleted] »

faragon wrote: post #6

Auto-vectorization is hard. Very hard. E.g. even in C/C++, the compiler (e.g. GCC C/C++ or MS VC/VC++) is unable to vectorize loops unless you help it a lot , and in most cases, you end writing SIMD "intrinsics" (e.g. [1]) in order to get optimal results. From my experience, despite auto-vectorization being better than 10 years ago, still is very far for optimizing code properly without lots of tuning (e.g. you can t…

[deleted]

Re: Java and SIMD

#20

Post by microcolonel »

stu2010 wrote: post #10
sargun wrote: post #9

Be very careful about this in a shared environment. AVX512 slows down the CPU cores because of thermal, voltage throttling. The instructions "take more work". Intel CPUs take 1 MILLIsecond to return to normal speed. If you're doing any kind of rapid context switching, or multiple workloads, depending on how your scheduler is setup, the OTHER workloads will show up as using more percentage of CPU time per work item. I…

Now that major cloud vendors are selling VMs with guaranteed AVX512 support, how are they going to deal with the "noisy neighbor" problem?

Giant heatsinks, and more sockets!

More likely though, they'll continue not to care. Performance is already hideously variable, it'll just get a bit worse.

Post reply on HN