21–30 of 34 posts
Given the problems with Java's floating-point implementation [1], would this be reliable for statistical analysis? [1] https://news.ycombinator.com/item?id=6585828
Given the problems with Java's floating-point implementation [1], would this be reliable for statistical analysis? [1] https://news.ycombinator.com/item?id=6585828
2. The math in FastR, if I understand the presentation correctly, is performed by FORTRAN libraries anyway. Using battle tested FORTRAN libraries for matrix computations is common practice in C, Java, Julia, Matlab and most other environments. They basically all share the same underlying matrix math code.
Earlier quoted context omitted.
> ... C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). As of 2002 (JDK 1.4) Java has excellent integration with native memory (you can freely pass pointers from C/FORTRAN to Java and vice versa[1]). There are numerous Java math libraries that use BLAS/LAPACK already[2]. In fact, AFAIK, most Java matrix math libraries u…
Ah, wonderful! So if I understand this correctly, this doesn't give C/Fortran access to Java-native primitive arrays; but instead, it's specific to NIO byte buffers (and then the matrix libraries have to build on top of that). But that should be fine for doing R replacements, at least in theory. (Personally, when programming Java I find it more convenient to use primitive arrays as opposed to matrix libraries, but th…
Direct byte buffers are very common in high-performance Java code. Reading/writing from/to those buffers can be made just as fast as plain Java arrays.
The programming language analysis is pretty interesting, but you have to ask, what's the point of a brand-new Java implementation? R isn't just a programming language, but it's a software framework/ecosystem. They mentioned this in the slides, but it's problematic because R crucially relies on C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory ac…
> ... C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). As of 2002 (JDK 1.4) Java has excellent integration with native memory (you can freely pass pointers from C/FORTRAN to Java and vice versa[1]). There are numerous Java math libraries that use BLAS/LAPACK already[2]. In fact, AFAIK, most Java matrix math libraries u…
I had used memory mapped buffers (which should be equivalent in performance), and there was no way to make the JIT inline access to these arrays. It was all calls (indirect, not properly branch predicted at that). Equivalent code to C++, running 10 times slower, with no way to speed it up.
(And the reason I was using memory maps, if you insist -is a 2GB read-only dataset used by multiple processes at the same time - I went to C++ eventually, because there was no way to get reasonable performance from Java, either memory use or speed. This is circa 2010)
Earlier quoted context omitted.
Ah, wonderful! So if I understand this correctly, this doesn't give C/Fortran access to Java-native primitive arrays; but instead, it's specific to NIO byte buffers (and then the matrix libraries have to build on top of that). But that should be fine for doing R replacements, at least in theory. (Personally, when programming Java I find it more convenient to use primitive arrays as opposed to matrix libraries, but th…
Java also gives native libraries direct access to primitive arrays, but that requires pinning them in place for the duration of the call (i.e. not letting the GC move them) so it incurs some performance penalty. Direct byte buffers are very common in high-performance Java code. Reading/writing from/to those buffers can be made just as fast as plain Java arrays.
Earlier quoted context omitted.
> ... C and Fortran interaction (which I thought the JVM can't do efficiently, since it doesn't like giving C/Fortran raw memory access to its internals). As of 2002 (JDK 1.4) Java has excellent integration with native memory (you can freely pass pointers from C/FORTRAN to Java and vice versa[1]). There are numerous Java math libraries that use BLAS/LAPACK already[2]. In fact, AFAIK, most Java matrix math libraries u…
Is there any way to ensure that accessing ByteBuffers (or whatever) is fast? I had used memory mapped buffers (which should be equivalent in performance), and there was no way to make the JIT inline access to these arrays. It was all calls (indirect, not properly branch predicted at that). Equivalent code to C++, running 10 times slower, with no way to speed it up. (And the reason I was using memory maps, if you insi…
For example, the Java Chronicle library[1] uses these techniques, as well as memory mapped files, to implement fast persistent message queues.
http://openjdk.java.net/projects/graal/
https://wiki.openjdk.java.net/display/Graal/Publications+and...
Given the problems with Java's floating-point implementation [1], would this be reliable for statistical analysis? [1] https://news.ycombinator.com/item?id=6585828
GNU R, for example, is implemented in C, and the implementations of R's basic arithmetic functions are actually quite complicated because they take care of so many of the edge cases cited in the cited post. For example, the round() function casts its argument first to a 64-bit before calling the C library's rint() function to preserve precision. [1]
Earlier quoted context omitted.
Is there any way to ensure that accessing ByteBuffers (or whatever) is fast? I had used memory mapped buffers (which should be equivalent in performance), and there was no way to make the JIT inline access to these arrays. It was all calls (indirect, not properly branch predicted at that). Equivalent code to C++, running 10 times slower, with no way to speed it up. (And the reason I was using memory maps, if you insi…
Yes. High performance Java code sometimes makes use of JVM intrinsics, accessible through the sun.misc.Unsafe class. Those are JITted down to a simple memory access instruction. That class also has intrinsics for CAS, and in JDK 8 it's got intrinsics for different memory fences as well. Those calls are compiled to a single native instruction. For example, the Java Chronicle library[1] uses these techniques, as well a…
Earlier quoted context omitted.
Yes. High performance Java code sometimes makes use of JVM intrinsics, accessible through the sun.misc.Unsafe class. Those are JITted down to a simple memory access instruction. That class also has intrinsics for CAS, and in JDK 8 it's got intrinsics for different memory fences as well. Those calls are compiled to a single native instruction. For example, the Java Chronicle library[1] uses these techniques, as well a…
Cool. Was this in Java 6? (circa 2010 - I couldn't find a way to do it back then). Also, why would you need (even on JDK7 or JDK8) unsafe access to jit a memory mapped access inline? Is there an underlying philosophical reason, or is it just that they never got to do it?
The sun.misc.Unsafe class is used extensively by JDK classes, and is meant for internal use. It provides intrinsics that are translated to a single machine instruction. Normally, you don't use the class directly. For example, you use, say, AtomicInt for CAS operations (which, internally uses s.m.Unsaafe) or the ByteBuffer class (which internally uses s.m.Unsafe for direct pointer access). The JDK classes add all sorts of protection (like range checks) around s.m.Unsafe, but if you know what you're doing, using s.m.Unsafe directly and eschewing some of those protections (usually adding ones more pertinent to your domain), you get some performance gains which may be significant depending on your use case.