A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video] • Hacker News Board

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#21

Post by chrisseaton » Sun, Jun 16, 2019, 6:08 PM UTC

pvg wrote: post #19

zlynx wrote: post #18

Earlier quoted context omitted.

A function could be compiled assuming the passed argument is always 2. All of the code for other values is just left out. As long as the compiled code has a check for values that are not 2 this code works great. It isn't correct though. At least, that's my interpretation.

Maybe I'm getting tripped up in the terminology here but to me this case is still a JIT jittin' - you look at runtime data and decide it's worth it to crank out a special case optimization for the input of 2. You produce that that optimization, which is sound along with a check to make sure it is applied only in the special case. You get to defer other optimization. The advantage here still seems to come from the run…

> to me this case is still a JIT jittin'

I think the point is that some JITs never do this kind of optimisation - they just produce the same code an AOT compiler would, but at runtime. Such as the .NET JIT.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#22

Post by ddragon » Sun, Jun 16, 2019, 6:09 PM UTC

pvg wrote: post #16

pron wrote: post #5

Earlier quoted context omitted.

JITs do more than just profile-guided optimizations. Their secret weapon is speculative optimizations that mean they don't need to work hard (and often fail) to prove the soundness of certain optimizations. They're allowed to guess and be wrong.

Wait, ignore soundness ? How does that work/provide advantage?

For example a JIT compiler can, based on the runtime information + inference, discard multiple branches of a function that are effectively pointless and possibly even just inline the results if given those inputs the function is deterministic.

Of course, the function it compiled will actually not work with any different but still valid arguments, but that's not really trouble since the JIT compiler will simply evaluate that the already compiled version won't work for those as the function is called and compile a new version of the function for the new types just before. A pure ahead-of-time compiler wouldn't be able to optimize so aggressively since it would lead to an exponential explosion of possible inputs combinations most of which will very likely never happen.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#23

Post by amelius » Sun, Jun 16, 2019, 6:11 PM UTC

chrisseaton wrote: post #20

amelius wrote: post #12

Earlier quoted context omitted.

If they do speculative optimizations, then doesn't that open the door to Spectre-like vulnerabilities, but now at the compiler level?

This isn't speculation as in speculative execution , it's speculation as in speculating that a condition is true that cannot be proved to be true , so it's not the same thing. An example of this kind of speculation is speculating that there will only ever be one thread in a system, and removing locks. If that speculation ever proves to be wrong - a second thread is created - the locks are put back into the system. Th…

Ok, but in a multi-user system (e.g. a webserver), if user 1 triggers a (de-)optimization, then user 2 can tell that a previous user was in that code path. Now I don't know how to extract useful information from that fact, but it shows that at least some information spills over the user boundaries.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#24

Post by pvg » Sun, Jun 16, 2019, 6:12 PM UTC

chrisseaton wrote: post #21

pvg wrote: post #19

Earlier quoted context omitted.

Maybe I'm getting tripped up in the terminology here but to me this case is still a JIT jittin' - you look at runtime data and decide it's worth it to crank out a special case optimization for the input of 2. You produce that that optimization, which is sound along with a check to make sure it is applied only in the special case. You get to defer other optimization. The advantage here still seems to come from the run…

> to me this case is still a JIT jittin' I think the point is that some JITs never do this kind of optimisation - they just produce the same code an AOT compiler would, but at runtime. Such as the .NET JIT.

I don't think that's the point the comment I'm replying to is making, or at least, it's not the point I'm asking about.

Edit: Your example in the other comment about the locks is the sort of thing I'm asking about. There, an optimization is made which is sound under some specific conditions and then unmade when those conditions change.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#25

Post by chrisseaton » Sun, Jun 16, 2019, 6:15 PM UTC

amelius wrote: post #23

chrisseaton wrote: post #20

Earlier quoted context omitted.

This isn't speculation as in speculative execution , it's speculation as in speculating that a condition is true that cannot be proved to be true , so it's not the same thing. An example of this kind of speculation is speculating that there will only ever be one thread in a system, and removing locks. If that speculation ever proves to be wrong - a second thread is created - the locks are put back into the system. Th…

Ok, but in a multi-user system (e.g. a webserver), if user 1 triggers a (de-)optimization, then user 2 can tell that a previous user was in that code path. Now I don't know how to extract useful information from that fact, but it shows that at least some information spills over the user boundaries.

Oh I see what you mean - Spectre-like rather than specifically Spectre. Yes I suppose specialisation (more generally than speculation) could leak information, in the same way as cache status can leak information.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#26

Post by pvg » Sun, Jun 16, 2019, 6:18 PM UTC

ddragon wrote: post #22

pvg wrote: post #16

Earlier quoted context omitted.

Wait, ignore soundness ? How does that work/provide advantage?

For example a JIT compiler can, based on the runtime information + inference, discard multiple branches of a function that are effectively pointless and possibly even just inline the results if given those inputs the function is deterministic. Of course, the function it compiled will actually not work with any different but still valid arguments, but that's not really trouble since the JIT compiler will simply evalua…

I put it in a bit more detail in a sibling comment but to me, there's no 'guessing about soundness and being wrong' in that particular scenario.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#27

Post by ddragon » Sun, Jun 16, 2019, 6:30 PM UTC

pvg wrote: post #26

ddragon wrote: post #22

Earlier quoted context omitted.

For example a JIT compiler can, based on the runtime information + inference, discard multiple branches of a function that are effectively pointless and possibly even just inline the results if given those inputs the function is deterministic. Of course, the function it compiled will actually not work with any different but still valid arguments, but that's not really trouble since the JIT compiler will simply evalua…

I put it in a bit more detail in a sibling comment but to me, there's no 'guessing about soundness and being wrong' in that particular scenario.

I guess it depends on the perspective and interpretation of soundness. If a JIT-compiler AoT compiles your entire program but infers that your function that implements logic that works for every number (as you defined it using the Number interface within the rules of it's type system) will only use 32 bits integers, then it will compile code that effectively does not hold up to the property that was established. The fact that it will stop execution the moment it reaches an invalid path and correct it doesn't change that.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#28

Post by twic » Sun, Jun 16, 2019, 6:53 PM UTC

pron wrote: post #11

exabrial wrote: post #6

I'm curious if GraalVM can do the same rockstar party tricks on Java 11? He uses Java 12, but doesn't go into huge detail why he chose that rather than the LTS.

12 is the current JDK version. OpenJDK has no notion of LTS (e.g. you will see no mention of 11 being an LTS on the project page: https://openjdk.java.net/projects/jdk/11/ nor any special designation compared to JDK 12's page), and all versions are equal. Java's LTS means something that could perhaps be quite different from LTS in other projects. LTS is a service offered by companies to arbitrary JDK versions of thei…

> OpenJDK has no notion of LTS

But two of the main maintainers, Oracle and Red Hat, absolutely do.

> LTS is a service offered by companies to arbitrary JDK versions of their choice

And that arbitrary version is 11.

It's technically accurate but substantially disingenuous to suggest that 11 is not Java's current LTS.

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#29

Post by pvg » Sun, Jun 16, 2019, 6:53 PM UTC

ddragon wrote: post #27

pvg wrote: post #26

Earlier quoted context omitted.

I put it in a bit more detail in a sibling comment but to me, there's no 'guessing about soundness and being wrong' in that particular scenario.

I guess it depends on the perspective and interpretation of soundness. If a JIT-compiler AoT compiles your entire program but infers that your function that implements logic that works for every number (as you defined it using the Number interface within the rules of it's type system) will only use 32 bits integers, then it will compile code that effectively does not hold up to the property that was established. The…

It could be and that would be uninteresting. But it's not hard to come up with a (contrived, limit-casey) optimization approach that does actually make guesses about soundness.

Let's say you wanted to optimize a short instruction sequence with a small domain of inputs. You could try to generate all (or at least, zillions) of similarly-sized possible instruction sequences and check them for soundness and performance. Now you're really making soundness guesses. Do real JITs actually make that sort of soundness guess (not that kind of attempt at optimization, obviously)?

Re: A Race of Two Compilers: GraalVM JIT versus HotSpot JIT C2 [video]

#30

Post by ddragon » Sun, Jun 16, 2019, 7:35 PM UTC

pvg wrote: post #29

ddragon wrote: post #27

Earlier quoted context omitted.

I guess it depends on the perspective and interpretation of soundness. If a JIT-compiler AoT compiles your entire program but infers that your function that implements logic that works for every number (as you defined it using the Number interface within the rules of it's type system) will only use 32 bits integers, then it will compile code that effectively does not hold up to the property that was established. The…

It could be and that would be uninteresting. But it's not hard to come up with a (contrived, limit-casey) optimization approach that does actually make guesses about soundness. Let's say you wanted to optimize a short instruction sequence with a small domain of inputs. You could try to generate all (or at least, zillions) of similarly-sized possible instruction sequences and check them for soundness and performance.…

A JIT compiler could detect that an instruction sequence (or function) is pure (for some range of valid inputs) and auto-memoize them for performance gains. But if you want the JIT to evolve compiled representation by profiling some fitness measurement (performance) and condition (soundness), that will most likely not happen any time soon. The JIT compiler has to balance compile time execution with runtime execution, if it wastes 5s to generate a program that runs in 2s when it could waste 1s to generate a program that would run in 4s then it would not be a good compiler at all. And above all else, JIT compilers, even the notoriously aggressive ones, still need to have some degree of predictability. If the user of the language can't predict the performance of the language, then they can't reliably improve their code performance.

I'm mostly talking about production-ready stuff, such work is certainly some fun playground. The Julia JIT (one of the notoriously aggressive JITs, for good and bad), allows users to, at runtime, add new context-aware behaviors to the compiler [1], and people used it for example to experiment with auto-parallelization of code and overall manipulating the code generated by the compiler. That was basically what got me into the language. So you could probably make a library that would inject some weird risky optimization that abuses the type system.

[1] https://docs.google.com/presentation/d/1IiBLVU5Pj-48vzEMnuYE...