Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

hal.inria.fr

Post reply on HN
Print view View original

11–20 of 56 posts

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#11

Post by mafribe »

astrobe_ wrote: post #10
vardump wrote: post #4

Earlier quoted context omitted.

While I somewhat agree with you, the real answer is assume nothing and profile . Always. But there's a catch. Things are getting a bit out of hand, there are just too many architectures and configurations. I think modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. Focus on 64-bit, but some attention should still b…

The fact that the performance of an implementation varies significantly depending on the actual CPU could already be deduced from some of the Ertl papers. Moreover, given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. "Optimizing" in this context simply means (almost) nothing. The obvious conclusion is to not care too muc…

    given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms.
Well, you could JIT the bytecode. Or you could use a meta-tracer (see e.g. [1]). Both (kind-of / sort-of) circumvent the platform dependency problem.

[1] http://tratt.net/laurie/research/pubs/html/bolz_tratt__the_i...

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#12

Post by vardump »

astrobe_ wrote: post #10
vardump wrote: post #4

Earlier quoted context omitted.

While I somewhat agree with you, the real answer is assume nothing and profile . Always. But there's a catch. Things are getting a bit out of hand, there are just too many architectures and configurations. I think modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. Focus on 64-bit, but some attention should still b…

The fact that the performance of an implementation varies significantly depending on the actual CPU could already be deduced from some of the Ertl papers. Moreover, given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. "Optimizing" in this context simply means (almost) nothing. The obvious conclusion is to not care too muc…

> you just rewrite the critical parts in native code.

Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore.

Pure interpreters are falling out of fashion anyways. For JITted systems, there's a significant cost for calling native code. At least until JITs actually inline natively called code, possibly even through dynamic library (.so, .dylib, .dll, etc.) call.

We need a lightweight, thin profile guided JIT/AOT engine. No standard library, memory management agnostic. Something that can target different architectures, memory & cache configurations and instruction set extensions that may have been unknown at design phase. A compilation target for some C/C++/Rust/etc. and a runtime that takes care of the architecture details.

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#13

Post by chrisdevereux »

Is there a reason that machine languages don't allow the programmer/compiler/runtime to explicitly control the CPU's cache & instruction pipeline?

Presumably they have access to much better information about the code's intent and future behaviour than the cpu does.

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#14

Post by loup-vaillant »

acqq wrote: post #5
vardump wrote: post #4

Earlier quoted context omitted.

While I somewhat agree with you, the real answer is assume nothing and profile . Always. But there's a catch. Things are getting a bit out of hand, there are just too many architectures and configurations. I think modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. Focus on 64-bit, but some attention should still b…

> the real answer is assume nothing and profile. They did profile, but what? Their "don't trust folklore" ("just 3% speedup") conclusion is valid only for one specific architecture and only for the most recent CPUs. As you say > modern consumer-oriented code should run at least well on Intel Nehalem - Skylake, AMD K10 - K12, ARM Cortex A7/A8/A9/A15/A53/A57/A72, Qualcomm Krait/Kryo. If I understand correctly, they the…

> Their "don't trust folklore" ("just 3% speedup") conclusion is valid only for one specific architecture and only for the most recent CPUs.

For now. In 5 years from now, you can be sure most new architectures will have predictors that are as good. So, while it might be good to reach for the low hanging fruit (jump threading), heavier approaches such as super-instruction replication are probably no longer a good idea.

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#15

Post by paulfr »

acqq wrote: post #3

If my coworker would come to me and show me his results where he measured that the "Labels as Values" implementation is not need while "on his Haswell CPU the speedup is only 3%" I'd just ask "can it be guaranteed that the code you propose to revert to the plain switch will run only on that CPU?" If not and the switch is in the performance sensitive place I'd keep the "Labels as Values" implementation. There are many…

> the one other researchers (which I respect much more) discovered in 2006

You seem to be blissfully unaware that they're in fact the same researchers: André Seznec is a co-author of the paper.

This paper is valuable in pointing out that ITTAGE branch prediction performance is a very good predictor of Haswell performance. Because the Haswell algorithm is secret, that should be very helpful to developers who still have to care about branch prediction.

Interpreter developers shouldn't ignore pre-Haswell chips just yet, but if you were to, say, develop a new language your design decisions should be guided by where the puck will be rather than where it used to be.

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#16

Post by [deleted] »

chrisdevereux wrote: post #13

Is there a reason that machine languages don't allow the programmer/compiler/runtime to explicitly control the CPU's cache & instruction pipeline? Presumably they have access to much better information about the code's intent and future behaviour than the cpu does.

[deleted]

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#17

Post by astrobe_ »

vardump wrote: post #12
astrobe_ wrote: post #10

Earlier quoted context omitted.

The fact that the performance of an implementation varies significantly depending on the actual CPU could already be deduced from some of the Ertl papers. Moreover, given that one uses specifically a bytecode interpreter and not a JIT is generally the sign that the interpreter is to be used on multiple platforms. "Optimizing" in this context simply means (almost) nothing. The obvious conclusion is to not care too muc…

> you just rewrite the critical parts in native code. Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore. Pure interpreters are falling out of fashion anyways. For JITted systems, there's a significant cost for calling native code. At least until JITs actually inline natively called code, possibly even through dynamic library (.so, .dylib, .dll, etc.) call. We ne…

> Rewrite critical parts of interpreter in native code? My point was there's no optimal native code anymore.

No, I mean rewrite the critical parts of the "interpreted" program as interpreter primitives/instructions: the best way to eliminate interpreter overhead is to merge N primitives into one.

But I realise we have a different perspective. You seem to consider interpreter VMs as off the shelf components, when I consider VMs as custom components because I'm working mainly with embedded systems. In the embedded world, JIT is often simply not an option either because they are not available for your particular target, or because the resource constrains don't allow it. But still you sometimes want to trade resources for ease of development and/or flexibility.

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#18

Post by sklogic »

chrisdevereux wrote: post #13

Is there a reason that machine languages don't allow the programmer/compiler/runtime to explicitly control the CPU's cache & instruction pipeline? Presumably they have access to much better information about the code's intent and future behaviour than the cpu does.

There are architectures that require manual pipeline control. MIPS, for example, or any VLIW architecture (including many of the modern GPUs). Cache is also often programmable (see scratch memory in Sparc, local memory in GPUs, etc.)

Re: Branch Prediction and the Performance of Interpreters – Don’t Trust Folklore

#19

Post by sklogic »

The article is missing one important difference between a primitive Python use of jump threading and the OCaml bytecode interpreter which pre-compiles the bytecode into a threaded code. A difference in performance between these two is huge.
Post reply on HN