Interesting articles, my only remark is not talking about JIT caches and PGO across process executions, but I guess it might come on a later post.
https://carolchen.me/blog/jits-intro/#pogo
It's there though not very in depth c:
A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
31–40 of 103 posts
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#32I just realized that the link color changes every time I load the page–I clicked on the article again and it was different from the last time I looked at it!
picking colours is really hard so I decided not to
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#33Earlier quoted context omitted.
> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even stat…
Except that PGO is in the box for most production quality JITs, the PGO data is even optimized between executions, while most developers never bother with using the PGO toolchains for languages like C and C++. Even if they lose in micro-benchemarks championships, it hardly matters in most enterprise codebases.
[deleted]
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#34Earlier quoted context omitted.
No unfortunately - it's a closed-source Enterprise feature I believe. But the current differential is pretty large and nobody is shouting loud that they can fix it using the PGO that I've heard. And what will the PGO determine that a JIT can't also do?
A number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get…
Most JIT compilers will go after any code that shows up as hot, regardless of when it executes. If your API code is that, even a minute after startup, it should really be getting optimized…
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#35Earlier quoted context omitted.
I too keep hearing this repeated but without empirical support. Same goes for the idea that garbage collection can be faster than manually managed memory. In practice, PGO has been the best possible compilation regime that I have ever found.
I think that’s my point. I get that JIT can produce faster code, I get that java is good enough for most things. But if you’re already in the performance critical regime there are other things you have to do where just having JIT available isn’t some magic bullet.
A possible magic solution is to load a JIT state from a previous execution and have good manual-optimization features to be like PGO. This would give JITs best of both worlds with being able to get to a strong peak performance and also not requiring manual tuning
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#36Earlier quoted context omitted.
but from a higher level.. you could just reimplement the thing in C++ with gcc and the whole thing will probably perform better. Basically what I’m saying is that I’ve never seen any substantial rewrite of decent C++ code into Java perform better, even if jitting has benefits on a small scale, it’s not substantial enough to overcome other overheads in managed languages.
C++ and Java have other language-level differences, though, that go beyond just "JITs are slower".
True, but part of that difference comes from the “we don't need to design the language for perf, the JIT will close the gap automagically” mindset though.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#37Earlier quoted context omitted.
C++ and Java have other language-level differences, though, that go beyond just "JITs are slower".
True, but part of that difference comes from the “we don't need to design the language for perf, the JIT will close the gap automagically” mindset though.
I don't really think so for Java; most of it was designed prior to high-performance JIT-based VMs were a thing. In fact I think a lot of the advancements in JITs actually came out of work that went into making HotSpot fast.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#38Earlier quoted context omitted.
Isn't this because Java is designed for JIT compilation, or at least not designed (with the appropriate tweaking knobs) for AOT compilation? Languages built with AOT compilation in mind (e.g. Rust or Nim) usually give you lots of ways make choices at compile time and give hints to the AOT compiler that the JIT compiler would instead try to infer at runtime in Java. But by infering these things at runtime intead, mayb…
Java has had commercial implementations of AOT compilers since the early 2000. Most compilers for embedded systems have always offered that option, and in what concerns enterprise JVMs, JIT compilers have had the capability to cache JIT code and PGO data between runs. Both options that have come now to OpenJDK, OpenJ9 and Graal. Android also learned the hard way that changing to pure AOT did not achieve the performan…
Pretty sure assymetrix was done in the late 90s. 1998 or so, they started out doing educational software and pivoted to doing ahead of time compilation for some weird reason. Here is a link to when it failed in 1999: https://www.cbronline.com/news/sepercede_exits_java_sells_to...
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#39Earlier quoted context omitted.
> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even stat…
I too keep hearing this repeated but without empirical support. Same goes for the idea that garbage collection can be faster than manually managed memory. In practice, PGO has been the best possible compilation regime that I have ever found.
> Same goes for the idea that garbage collection can be faster than manually managed memory.
It's really workload dependent and depend a lot of the GC involved (a pretty dumb one like Python's or Go's won't get you anything performance wise), but a copying collector can achieve allocation way faster than a regular heap allocator (the allocation can be almost as cheap as allocating on the stack). If you can't avoid boxing and your objects aren't all long-lived, you can run circles around a program not using such a GC.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#40Earlier quoted context omitted.
except Julia because Julia <3
Or .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.
Some people make a distinction between 'just-in-time compilation' (like .NET - it's normal static compilation, but done just before first execution) and 'dynamic compilation' (like Java, it choses when to compile, uses dynamic information, may re-compile, etc.)