Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

JIT: So you want to be faster than an interpreter on modern CPUs

pinaraf.info

Post reply on HN
Print view View original

51–60 of 66 posts

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#51

Post by gary_0 »

Arnavion wrote: post #11
gary_0 wrote: post #9

> This is called branch prediction, it has been the source of many fun security issues... No, that's speculative execution you just described. Branch prediction was implemented long before out-of-order CPUs were a thing, as you need branch prediction to make the most of pipelining (eg. fetching and decoding a new instruction while you're still executing the previous one--if you predict branches, you're more likely to…

Speculative execution does not require out-of-order execution. When you predict a branch, you're speculatively executing the predicted branch. Whether you're doing it in the same order as instruction order or out of order is independent of that.

The article is talking about OoO which is why I mentioned it. My point is that branch prediction and speculative execution are different things. You can do speculative execution without a branch predictor (run both branches and throw out the one that's wrong).

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#52

Post by gary_0 »

Wowfunhappy wrote: post #33
Arnavion wrote: post #11

Earlier quoted context omitted.

Speculative execution does not require out-of-order execution. When you predict a branch, you're speculatively executing the predicted branch. Whether you're doing it in the same order as instruction order or out of order is independent of that.

If you're executing instructions in order, wouldn't you already know the result of the branch by the time you reach its code?

A good explanation of branch prediction: https://danluu.com/branch-prediction/

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#53

Post by pcwalton »

duped wrote: post #42
bencyoung wrote: post #25

Earlier quoted context omitted.

JIT compilation can be faster for compiled languages too, as it allows data driven inlining and devirtualization, as well as "effective constant" propogation and runtime architecture feature detection

Is there a production JIT for a compiled language that is actually faster? I understand the theory, I don't think the practice backs it up.

I believe HotSpot is usually faster than GCJ.

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#54

Post by pcwalton »

varjag wrote: post #45
fragmede wrote: post #44

Earlier quoted context omitted.

Depends, what do you consider Java?

Java is certainly not the fastest language out there.

Sure, but the relevant comparison isn't between languages: it's between a state-of-the-art JIT implementation of one language and a likewise-state-of-the-art AOT implementation of the same language. Unfortunately there aren't many examples of this; most languages have a preferred implementation strategy that receives much more effort than the other one.

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#55

Post by Arnavion »

gary_0 wrote: post #51
Arnavion wrote: post #11

Earlier quoted context omitted.

Speculative execution does not require out-of-order execution. When you predict a branch, you're speculatively executing the predicted branch. Whether you're doing it in the same order as instruction order or out of order is independent of that.

The article is talking about OoO which is why I mentioned it. My point is that branch prediction and speculative execution are different things. You can do speculative execution without a branch predictor (run both branches and throw out the one that's wrong).

You're right, I missed the article specifically mentions Meltdown in that sentence, not Spectre.

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#56

Post by stmw »

stmw wrote: post #3

Good read. But a word of caution - the "JIT vs interpreter" comparisons often favor the interpreter when the JIT is inplemented as more-or-less simple inlining of the interpreter code. (Here called "copy-and-patch" but a decades-only approach). I've had fairly senior engineers try to convince me that this is true even for Java VMs. It's not in general, at least not with the right kind of JIT compiler design.

Turns out one of the classic papers on this is available for those interested in this discussion - https://news.ycombinator.com/item?id=45582127

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#57

Post by Wowfunhappy »

gf000 wrote: post #50
cmeacham98 wrote: post #40

Earlier quoted context omitted.

Source? Is there any non-Apple app that has this entitlement?

I believe the Delta emulator has JIT support, but possibly only when installed as a developer.

As far as I can tell, you need to connect your phone to a PC running software which enables JIT by exploiting a feature intended for remote debugging. https://faq.altstore.io/altstore-classic/enabling-jit

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#58

Post by trashface »

Doesn't work for every case, but I think for a lot of cases nowadays, if you are using an interpreter and its slow, you should just generate web assembly. Libraries like walrus for rust make this pretty easy to do, and wasmtime provides a serviceable standalone runtime. For my little language, recursive fib(40) executes in firefox with wasm in about 600ms. My interpreter basically can't finish it.

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#59

Post by stmw »

gr4vityWall wrote: post #2

That was a pretty interesting read. My take is that you can get pretty far these days with a simple bytecode interpreter. Food for thought if your side project could benefit from a DSL!

As I noted in another comment, I would like to persuade you that this is not the right take-away..

Re: JIT: So you want to be faster than an interpreter on modern CPUs

#60

Post by stmw »

imtringued wrote: post #7

I'm not really interested in building an interpreter, but the part about scalar out of order execution got me thinking. The opcode sequencing logic of an interpreter is inherently serial and an obvious bottleneck (step++; goto step->label; requires an add, then a fetch and then a jump, pretty ugly). Why not do the same thing the CPU does and fetch N jump addresses at once? Now the overhead is gone and you just need t…

The thing you're suggesting makes sense, but it's far more efficient to do in hardware. You might say that you could do it on one of the many cores available on your modern processor, but it turns out that synchronizing them to your main thread is really inefficient -- and anyway, they're busy running your HN browser threads and your YouTube music video.
Post reply on HN