It seems kinda unfair to compare 32bit ARMv7 against 64bit x86-64. Wouldn't it be much fairer to compare ARMv7 against the register-starved x86, or to compare AArch64 against x86-64?
Not to mention clock and cache differences.
ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
11–20 of 120 posts
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#12Which version of OCaml? The ARMv7 backend was rewritten about 2 years ago, and merged in 4.00. http://caml.inria.fr/mantis/view.php?id=5433 The new backend is supposed to be considerably faster on floating point code. This code looks integer only, and I don't have relative performance of old/new backend for integer code. As a wider question: Who cares much about ARMv7? ARMv8 is a completely different beast, requiring…
OCaml version 4.02.0. I don't have an ARMv8 device to benchmark on, unfortunately.
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#13Which version of OCaml? The ARMv7 backend was rewritten about 2 years ago, and merged in 4.00. http://caml.inria.fr/mantis/view.php?id=5433 The new backend is supposed to be considerably faster on floating point code. This code looks integer only, and I don't have relative performance of old/new backend for integer code. As a wider question: Who cares much about ARMv7? ARMv8 is a completely different beast, requiring…
Everyone cares about armv7 because it is the CPU architecture deployed in over 95% of the mobile devices.
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#14The LuaJIT results don't surprise me. I've always been impressed with LuaJIT. The OpenJDK results also don't surprise me. If you work in a Java shop, you learn very quickly to throw out OpenJDK in favor of Sun/Oracle Java. OpenJDK is indeed a "steaming pile of crap". However, I would have liked to see Julia and Javascript benchmarks in those results. I've heard great things about Julia, and knowing just how incredibl…
It seems to me though, that the Java numbers aren't representative. As far as I know, JVM benchmarks should allow HotSpot etc to optimise the functions by calling them ~1k times before the actual benchmark happens. That doesn't seem to be the case in the code.
The same function is being called over and over millions of times, so hopefully that should give HotSpot a chance to kick in. I didn't want to add artificial warm up as I don't think that's representative of real code.
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#15The LuaJIT results don't surprise me. I've always been impressed with LuaJIT. The OpenJDK results also don't surprise me. If you work in a Java shop, you learn very quickly to throw out OpenJDK in favor of Sun/Oracle Java. OpenJDK is indeed a "steaming pile of crap". However, I would have liked to see Julia and Javascript benchmarks in those results. I've heard great things about Julia, and knowing just how incredibl…
>OpenJDK is indeed a "steaming pile of crap" Just to clarify, it performed as well as the Oracle JVM on x86. Its poor performance on ARM is just due to its lack of JIT compilation. >However, I would have liked to see Julia and Javascript benchmarks in those results. I'm happy to include Javascript or Julia implementations if someone supplies them. I wasn't comfortable with Julia enough to write one myself.
>> OpenJDK is indeed a "steaming pile of crap"
> Just to clarify, it performed as well as the Oracle JVM on x86. Its poor performance on ARM is just due to its lack of JIT compilation.
Understood. Admittedly, I'm a system administrator first, developer second. However, my experience has shown that Sun/Oracle JVM usually out performs OpenJDK. Even in development, on the Java teams I've had to support, OpenJDK is never preferred or wanted.
Further, RHEL ships the OpenJDK compiled with the GNU compiler for Java (GCJ), as well as GNU's classpath. From what I've seen supporting these Java teams, and not being a Java developer, it's unstable and slow.
So, in this specific instance, OpenJDK may have performed as well as the Sun/Oracle JVM on x86-64, but in the broader scope, it doesn't seem to hold up. Just my experiences though. Take it with a grain of salt.
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#16It seems kinda unfair to compare 32bit ARMv7 against 64bit x86-64. Wouldn't it be much fairer to compare ARMv7 against the register-starved x86, or to compare AArch64 against x86-64?
It's not comparing the architectures against each other, it's comparing language implementations against each other on those architectures. As x86-64 is the most common x86 architecture nowadays, and ARMv7 the most common ARM architecture (at least on mobile), comparing language's performance on them should hopefully be more relevant as more developers use those platforms.
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#17Which version of OCaml? The ARMv7 backend was rewritten about 2 years ago, and merged in 4.00. http://caml.inria.fr/mantis/view.php?id=5433 The new backend is supposed to be considerably faster on floating point code. This code looks integer only, and I don't have relative performance of old/new backend for integer code. As a wider question: Who cares much about ARMv7? ARMv8 is a completely different beast, requiring…
Except it's really hard to benchmark on ARMv8 since non-Apple hardware is expensive.
Big companies have plenty of access. I work for Red Hat and have one right next to my desk. I also know SUSE and MSFT have the hardware, and I'm sure many more.
I've been working on fixing bugs in the ARMv8 backend in the OCaml compiler - it's now stable, and performs very well (I can't go into exact performance details for contract reasons).
http://caml.inria.fr/mantis/view.php?id=6489
http://caml.inria.fr/mantis/view.php?id=6284
http://caml.inria.fr/mantis/view.php?id=6283
(and more ...)
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#18This is some quality work. The Java results are particularly striking! It would be interesting to see some more languages that have more than one implementation compared. What is the difference between FSharp and F#?
FSharp instead of F# is just a way of getting around issues with using the # character in the shell scripts that run the benchmark.
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#19Obligatory question: what compiler options did you use for c++? Good first try is -o3 -march=native. It really makes huge difference in case of C/C++
Re: ARMv7 vs. x86-64: Pathfinding benchmark of C++, D, Go, Nim, Ocaml, and more
#20That's some alien Java code indeed.