I just realized that the link color changes every time I load the page–I clicked on the article again and it was different from the last time I looked at it!
A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
21–30 of 103 posts
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#22Earlier quoted context omitted.
> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even stat…
I too keep hearing this repeated but without empirical support. Same goes for the idea that garbage collection can be faster than manually managed memory. In practice, PGO has been the best possible compilation regime that I have ever found.
I think that’s my point. I get that JIT can produce faster code, I get that java is good enough for most things. But if you’re already in the performance critical regime there are other things you have to do where just having JIT available isn’t some magic bullet.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#23Earlier quoted context omitted.
> in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application Today you make make a very direct empirical comparison to see this - using the Graal compiler. This lets you compile exactly the same Java code either ahead-of-time or just-in-time, but using the same compiler logic except for the runtime information available when running jus…
Isn't this because Java is designed for JIT compilation, or at least not designed (with the appropriate tweaking knobs) for AOT compilation? Languages built with AOT compilation in mind (e.g. Rust or Nim) usually give you lots of ways make choices at compile time and give hints to the AOT compiler that the JIT compiler would instead try to infer at runtime in Java. But by infering these things at runtime intead, mayb…
Java has had commercial implementations of AOT compilers since the early 2000.
Most compilers for embedded systems have always offered that option, and in what concerns enterprise JVMs, JIT compilers have had the capability to cache JIT code and PGO data between runs.
Both options that have come now to OpenJDK, OpenJ9 and Graal.
Android also learned the hard way that changing to pure AOT did not achieve the performance improvements that they expected, while compilation on device achieved C++ compile times when it was time to update all apps, hence the multi-tier interpreter/JIT/AOT with PGO introduced in Android 7.
The main problem of AOT compilation with PGO, is that first of all one needs a good dataset so that the optimizations are in line with the actual behaviour in production, still doesn't work across dynamic libraries so optimizations like devirtualization are not possible, and most of the time the tooling is quite cumbersome to use.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#24Earlier quoted context omitted.
> in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application Today you make make a very direct empirical comparison to see this - using the Graal compiler. This lets you compile exactly the same Java code either ahead-of-time or just-in-time, but using the same compiler logic except for the runtime information available when running jus…
but from a higher level.. you could just reimplement the thing in C++ with gcc and the whole thing will probably perform better. Basically what I’m saying is that I’ve never seen any substantial rewrite of decent C++ code into Java perform better, even if jitting has benefits on a small scale, it’s not substantial enough to overcome other overheads in managed languages.
C++ and Java have other language-level differences, though, that go beyond just "JITs are slower".
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#25Tl;dr: JIT is a compiler that optimuzes certain parts of code after interpreter see it as hot (e.g. was executed many time). Thank to runtime information it can occasionally exceed performance of statically compiled language.
except Julia because Julia <3
Or .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#26Interesting articles, my only remark is not talking about JIT caches and PGO across process executions, but I guess it might come on a later post.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#27Tl;dr: JIT is a compiler that optimuzes certain parts of code after interpreter see it as hot (e.g. was executed many time). Thank to runtime information it can occasionally exceed performance of statically compiled language.
> Thank to runtime information it can occasionally exceed performance of statically compiled language. Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole... amdahls law and all that. You can always hand tune the 1-10% hotspots for reasonable cost most of the time, and even stat…
JITs vs static compilers on a language made for static compilers is probably a contest that JITs will never win. The keys to making JITs "better" than static compilation is (by my limited knowledge) creation of languages that are made to be JIT compiled (like Java) and ensuring that tuning JITs is easier than PGO (though nothing that is currently in development seems promising to this end)
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#28Earlier quoted context omitted.
except Julia because Julia <3
Or .Net, which entirely lacks an interpreter, and JIT compiles everything that it executes. (Like Julia, if I understand correctly.) JIT != mixed-mode execution.
Mono has one, AFAIK: https://www.mono-project.com/news/2017/11/13/mono-interprete...
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#29Earlier quoted context omitted.
Have you tried graal's PGO?
No unfortunately - it's a closed-source Enterprise feature I believe. But the current differential is pretty large and nobody is shouting loud that they can fix it using the PGO that I've heard. And what will the PGO determine that a JIT can't also do?
A number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get much optimization.
With PGO, I can get a more representative profiling dataset, allowing the JIT to see actual production loads instead of startup loads.
And for CLI apps, PGO code starts up fast and never slows down to profile or optimize.
Re: A Deep Introduction to JIT Compilers: JITs are not very Just-in-time
#30Earlier quoted context omitted.
No unfortunately - it's a closed-source Enterprise feature I believe. But the current differential is pretty large and nobody is shouting loud that they can fix it using the PGO that I've heard. And what will the PGO determine that a JIT can't also do?
A number of things actually. For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup, as opposed to steady state operation. For example, I have a Play app that takes a minute to start up. The JIT does a great job of optimizing the code that is called during the setup process, but the API code itself doesn't get…
> For one, most JIT implementations only optimize once, instead of continuously. The result is machine code that is optimized for the sorts of things done at startup
Funny you should mention that - the author of this blog post has another post on fixing that problem for one specific (but very practical) case where we want to disregard some profiling information from the startup phase because it pollutes the genuine profiling information.
https://engineering.shopify.com/blogs/engineering/optimizing...