Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

tratt.net

Post reply on HN
Print view View original

21–30 of 93 posts

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#21

Post by anp »

pizlonator wrote: post #14
Spivak wrote: post #11

Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say dumping some data structure when it hits steady state to give hints to the JIT the next time it runs?

That would be great but it’s hard because the trick, at least in JS VMs, is to have the JIT specialize based on the heap. So you’d need a heap snapshot or some way to link the generated code to a different heap. Maybe not impossible, just hard enough that it’s not widespread.

IIRC V8 can now provide deterministic JIT+heap images in addition to preparsed bytecode images.

EDIT: worth noting that other JS VMs like JSC have these APIs for bytecode too, but i can’t remember off hand whether they can do generated code too.

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#22

Post by rovr138 »

hamburglar wrote: post #17
pizlonator wrote: post #14

Earlier quoted context omitted.

That would be great but it’s hard because the trick, at least in JS VMs, is to have the JIT specialize based on the heap. So you’d need a heap snapshot or some way to link the generated code to a different heap. Maybe not impossible, just hard enough that it’s not widespread.

It does seem like you could do JIT with a persistent cache which stores the JIT output along with a key that's a hash of all the relevant system parameters like CPU model and VM parameters like heap size. This would mean that the typical case of re-running a program in the same environment would be pre-warmed.

Kind of like memchached, https://en.wikipedia.org/wiki/Memcached

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#23

Post by pizlonator »

hamburglar wrote: post #17
pizlonator wrote: post #14

Earlier quoted context omitted.

That would be great but it’s hard because the trick, at least in JS VMs, is to have the JIT specialize based on the heap. So you’d need a heap snapshot or some way to link the generated code to a different heap. Maybe not impossible, just hard enough that it’s not widespread.

It does seem like you could do JIT with a persistent cache which stores the JIT output along with a key that's a hash of all the relevant system parameters like CPU model and VM parameters like heap size. This would mean that the typical case of re-running a program in the same environment would be pre-warmed.

It’s much harder than that because the JIT is speculating on what lots of objects in the heap are doing, including watchpointing them to constant fold properties. It’s not clear what the key should be in that case.

Still not impossible but I want to be clear on what exactly makes this hard. CPU model for example is not what makes it hard.

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#24

Post by pizlonator »

ddevault wrote: post #10

It occurs to me that, in practical terms, the "steady state" of performance with the increasingly large blobs of JavaScript we find littered throughout the web tends to impact the user more than other JITs. As the user navigates from page to page, the VM is reset, a fresh set of minified blobs is downloaded and JITed, and a core or two is pinned to do so. That translates pretty directly to a hotter phone, less batter…

JS VMs try to be really good and smart about startup time like when you visit a page for the first time.

The main trick is to interpret the short running code.

That said, some JITing makes page load faster. But only if the JIT only kicks in for functions that run more than some amount of time and the JIT is very cheap to run.

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#25

Post by evmar »

pizlonator wrote: post #2

It would be better if VM comparisons included JSC rather than, or in addition to, V8. JSC tends to outperform V8 so if you find a pathology in V8 it’s just not so surprising. It would be more interesting if you found a pathology in JSC. I think that the use of small benchmarks obscures what’s going on. The VM is trying to win in the average. It’s like a professional gambler. Observing that the VM did something dumb f…

This blog post, and your reply, both touch on how difficult it is to measure performance. But then in the same where you point out that there are many different valid ways you could measure performance, you also make the broad claim that "JSC tends to outperform V8". What are you basing that on?

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#26

Post by cortesoft »

Spivak wrote: post #11

Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say dumping some data structure when it hits steady state to give hints to the JIT the next time it runs?

Isn't that just compiling, then?

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#27

Post by pizlonator »

evmar wrote: post #25
pizlonator wrote: post #2

It would be better if VM comparisons included JSC rather than, or in addition to, V8. JSC tends to outperform V8 so if you find a pathology in V8 it’s just not so surprising. It would be more interesting if you found a pathology in JSC. I think that the use of small benchmarks obscures what’s going on. The VM is trying to win in the average. It’s like a professional gambler. Observing that the VM did something dumb f…

This blog post, and your reply, both touch on how difficult it is to measure performance. But then in the same where you point out that there are many different valid ways you could measure performance, you also make the broad claim that "JSC tends to outperform V8". What are you basing that on?

These days we use JetStream 2 (our design) and Speedometer 2 (collaborative design between WK and Chromium folks) as the main big benchmarks but it’s not the only thing we measure and tune.

V8 used to have their own JS benchmark, Octane, but they retired it at about the same time as we beat them on it. So JSC is fast enough to make other people retire their benchmarks.

And by the way if you are interested in what we think of as good methodology you should read about JetStream 2: https://webkit.org/blog/8685/introducing-the-jetstream-2-ben...

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#28

Post by dodobirdlord »

cortesoft wrote: post #26
Spivak wrote: post #11

Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say dumping some data structure when it hits steady state to give hints to the JIT the next time it runs?

Isn't that just compiling, then?

Sure, but something like Java already has to be compiled. Throw a few more minutes in there, maybe run the test suite a couple thousand times.

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#29

Post by jakear »

Spivak wrote: post #11

Stupid question for people who know more about these things. Why can’t we fight the warmup time by running an already warmed up snapshot of the program? Or say dumping some data structure when it hits steady state to give hints to the JIT the next time it runs?

I believe this is what Deno tries to do.

https://deno.land/

Re: Why Aren’t More Users More Happy With Our VMs? Part 1 (2018)

#30

Post by pizlonator »

jjnoakes wrote: post #18
hansvm wrote: post #9

Earlier quoted context omitted.

I mostly agree, but compilers intentionally don't explore every optimization path to save compilation time, and a JIT is able to trace the program for real data and find places that could benefit from additional attention. I don't think JITs are the only way to address that trade-off in compilation time vs runtime performance metrics, and whether their benefits are worth their other costs is an interesting, program-d…

Any languages explore both? Compile to native and run with a jit which profiles and optimizes further based on runtime behavior?

No fundamental reason why you cannot do this. Maybe Graal can even do this for Java since it can both compile to native and do profile guided optimization (but I don't know if they ever combine the two).

Here's a reason why this might not be a super great thing to do, if you're thinking of it from a language design standpoint:

- Profiling and rejitting costs you memory and makes all code pay a tax. The biggest tax is the safepoint/OSR tax. It costs significant memory and some small amount of time (maybe the time cost you add from comprehensive OSR support is like 1%-5% overall, but I'm not sure, because it's hard to isolate this cost and measure it). So you don't want to build a system that supports JITing unless it's going to give you big wins. That implies languages with lots of dynamic typing.

- The languages that most benefit from JITs (the win they get consistently overcomes the overhead of JITing) are the ones that have so much dynamic typing features that compiling them to native is a super hard problem.

Java is one of the few languages that is both dynamic enough to benefit from JITs but static enough to be possible to compile to native. And even for Java the native compilation is a lot of effort to get right.

Note that "possible to compile to native" to me means: compiling to native produces something that performs well so there exists some benefit to actually doing it. Like, I would expect compiling JavaScript to native to produce something that doesn't perform well at all.

So, basically, the issue is that most languages are either in the "benefit of JIT is smaller than cost of JIT" category because they have adequately static typing or in the "must have JIT and cannot compile statically" category because they don't have static typing. Not a lot of languages are in the sweet spot where doing both would help, but such languages exist (Java) and there's no reason why they can't do what you suggest and they may already do it (seems like a natural thing for Graal to do).

Post reply on HN