Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

carolchen.me

Post reply on HN
Print view View original

41–50 of 103 posts

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#41

Post by dialamac »

saagarjha wrote: post #37
littlestymaar wrote: post #36

Earlier quoted context omitted.

True, but part of that difference comes from the “we don't need to design the language for perf, the JIT will close the gap automagically” mindset though.

I don't really think so for Java; most of it was designed prior to high-performance JIT-based VMs were a thing. In fact I think a lot of the advancements in JITs actually came out of work that went into making HotSpot fast.

Sure but one of vocal pitches/hype for Java in the beginning and even smalltalk before that.. was don’t worry about the price paid for .. automatic memory management, bytecode, dynamism... the top men are working on JIT, GC research and other optimizations that would not only close, but handily overtake the gap compared to the state of the art static compiled languages of the time. (Hence exactly why Hotspot was a thing after java was clearly becoming widespread)

In reality it was a bit of a mixed bag.. and to some of us that remember the hype from 30 years ago it comes across as over promising and underdelivering.

That isn’t to say that the technology isn’t incredible, I don’t mean to dump on it. But overpromising is sort of the status quo for tech.

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#42

Post by darksaints »

saagarjha wrote: post #34
darksaints wrote: post #29

Earlier quoted context omitted.

A number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get…

Most JIT compilers will go after any code that shows up as hot, regardless of when it executes. If your API code is that, even a minute after startup, it should really be getting optimized…

Think of an use case where you have an API whose mode behavior is to do nothing, waiting for a call... but commonly gets calls that are very compute intense for short bursts of time.

With the most common type of JITs, which profile once and compile once, I'm going to get code that is optimized for startup and initialization.

If I have an "advanced" JIT, which is constantly deoptimizing and reoptimizing for whatever it sees as the hottest path for some arbitrarily chosen snapshot of time, I'm going to see my compute-intensive code slowed down so that it can optimize and compile it every time that endpoint is called, but then subsequently deoptimized while it is sitting around waiting for something to do, ensuring that I have to go through the same process the next time it is called. You can actually see a lot of situations where this regime could be even worse than a naive startup-based single optimization, which is why it is actually not that common outside of dynamically typed languages.

With PGO, I can select a profile snapshot during a stress test, and get heavily optimized code specifically for the things that actually stress the server. And it will stay optimized for that use case.

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#43

Post by joe_the_user »

This article is really nice for not making things harder than they needed to be. Trace then substitute the trace - no magic.

It seems like a big part of all these strategies is making sure the checks for the optimizations (checking if they're possible, checking if they're done) don't take more time than the optimizations save.

Or could there be a way to alter the execution graph so that once you add in the optimization, you never have to check if it's there?

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#44

Post by vlovich123 »

Is anyone aware of any efforts to get clang/rustc to use lli to speed up iteration times? My thought is you lower everything to llvm bytecode first but skip all the codegen/linking. Then you can codegen or run interpreted. Might help make debug builds much faster to get to execution. Probably could trivially parallelize the background compilation & auto-build in the background on every file change to get the best of all worlds (most code gen is ready, the remainder is executable & will get compiled as needed). If you can do it with live substitution (which I think is what lli does) then you'd progressively get a faster program.

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#45

Post by ddragon »

chrisseaton wrote: post #40
MaxBarraclough wrote: post #25

Earlier quoted context omitted.

Or .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.

Some people make a distinction between 'just-in-time compilation' (like .NET - it's normal static compilation, but done just before first execution) and 'dynamic compilation' (like Java, it choses when to compile, uses dynamic information, may re-compile, etc.)

In the Julia community they frequently call it Just Ahead-Of-Time the compiler strategy of making the inference of all downstream types and methods to dispatch and compiling all at once. And the period while purely running a static program "world age", which usually lasts until you go back to the global namespace or you need any dynamic feature. Julia does not have an interpreter outside of the debugger.

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#46

Post by pjmlp »

kipply wrote: post #31
pjmlp wrote: post #26

Interesting articles, my only remark is not talking about JIT caches and PGO across process executions, but I guess it might come on a later post.

https://carolchen.me/blog/jits-intro/#pogo It's there though not very in depth c:

This is not what I am talking about, JIT caches with PGO data work in a different way.

https://openjdk.java.net/jeps/310

https://www.eclipse.org/openj9/docs/xcodecachetotal/

https://docs.oracle.com/cd/E13188_01/jrockit/docs142/usergui...

https://source.android.com/devices/tech/dalvik/jit-compiler#...

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#47

Post by darksaints »

littlestymaar wrote: post #39
darksaints wrote: post #12

Earlier quoted context omitted.

I too keep hearing this repeated but without empirical support. Same goes for the idea that garbage collection can be faster than manually managed memory. In practice, PGO has been the best possible compilation regime that I have ever found.

> Same goes for the idea that garbage collection can be faster than manually managed memory. It's really workload dependent and depend a lot of the GC involved (a pretty dumb one like Python's or Go's won't get you anything performance wise), but a copying collector can achieve allocation way faster than a regular heap allocator (the allocation can be almost as cheap as allocating on the stack). If you can't avoid bo…

Yes, allocation can be incrementally be made cheaper with a copying GC. But you have to realize that it still comes at a cost, and allocation speed isn't always a bottleneck. You have a runtime process intercepting your code state, scanning and analyzing your heap and making a decision to keep, copy, or free. For anything larger than a trivially small object graph, that additional processing can easily overwhelm any incremental allocation speed benefit that you get.

I get that it is theoretically possible for GC to be faster, but I literally have never seen it pan out in practice.

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#48

Post by kipply »

pjmlp wrote: post #46
kipply wrote: post #31

Earlier quoted context omitted.

https://carolchen.me/blog/jits-intro/#pogo It's there though not very in depth c:

This is not what I am talking about, JIT caches with PGO data work in a different way. https://openjdk.java.net/jeps/310 https://www.eclipse.org/openj9/docs/xcodecachetotal/ https://docs.oracle.com/cd/E13188_01/jrockit/docs142/usergui... https://source.android.com/devices/tech/dalvik/jit-compiler#...

I don't think I'm familiar with this (I may have come across it but didn't identify it by name when I worked with Graal?), could you provide a link? :o Thanks!

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#49

Post by pjmlp »

kipply wrote: post #48
pjmlp wrote: post #46

Earlier quoted context omitted.

This is not what I am talking about, JIT caches with PGO data work in a different way. https://openjdk.java.net/jeps/310 https://www.eclipse.org/openj9/docs/xcodecachetotal/ https://docs.oracle.com/cd/E13188_01/jrockit/docs142/usergui... https://source.android.com/devices/tech/dalvik/jit-compiler#...

I don't think I'm familiar with this (I may have come across it but didn't identify it by name when I worked with Graal?), could you provide a link? :o Thanks!

I have provided links above, there are also a couple of Java Language Summit, Code/Java ONE and Google IO talks about them, but I would need to search for them. Can provide them later.

Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time

#50

Post by kipply »

pjmlp wrote: post #49
kipply wrote: post #48

Earlier quoted context omitted.

I don't think I'm familiar with this (I may have come across it but didn't identify it by name when I worked with Graal?), could you provide a link? :o Thanks!

I have provided links above, there are also a couple of Java Language Summit, Code/Java ONE and Google IO talks about them, but I would need to search for them. Can provide them later.

Thanks, I know of this stuff but I don't think I would've been qualified to form an explanation/overview of them. It would be nice to have these in a post though!
Post reply on HN