71–80 of 103 posts
Earlier quoted context omitted.
The one problem with LLVM is that it’s not very well suited to JIT compilers, a major component of which is its slowness.
That’s not really accurate. It has strict requirements about stack traversal required to appropriately trace memory roots. This is incidental to JIT vs AOT.
Earlier quoted context omitted.
Im very skeptical that the comparison really makes sense. Programs written in GC languages are so different in their allocation behavior from those written in manual memory languages that I don't think you can meaningfully compare them. And comparing regular C++ with Boehm GC is probably not that interesting, as Boehm is extremely limited by the semantics of C++. And whichever way you look at it, the GC doesn't reall…
> And whichever way you look at it, the GC doesn't really have to outperform manual memory management to be extremely useful. It just has to be close enough, the correctness guarantees alone make it worth it as long as the performance difference is not too large, in many domains, not to mention the productivity boost and program readability. I agree with all of this. But it is a very different argument than the theor…
Most likely if you compare a Rust and Java program that both run a loop creating a million node linked list and then starting over, you'll see that the Java program is much faster. I'll actually try to do that and confirm, but I believe it should be the case.
Even if you were to allocate arrays (Vec) you may see the same.
But, of course, that is not a fair comparison. Rust gives you tools to avoid allocation in the first place. It also optimizes so many things that Java doesn't. Summing up all of the values in an ArrayList in Java is going to be so much slower than doing the same in Rust it's not even funny. I'm not even sure that Java implements vectorization yet.
It would be very interesting to see what performance a Rust program would get with a good, concurrent parallel generational copying collector. Perhaps in a few years there will be some interest in that.
Earlier quoted context omitted.
The one problem with LLVM is that it’s not very well suited to JIT compilers, a major component of which is its slowness.
That’s not really accurate. It has strict requirements about stack traversal required to appropriately trace memory roots. This is incidental to JIT vs AOT.
Earlier quoted context omitted.
> Same goes for the idea that garbage collection can be faster than manually managed memory. It's really workload dependent and depend a lot of the GC involved (a pretty dumb one like Python's or Go's won't get you anything performance wise), but a copying collector can achieve allocation way faster than a regular heap allocator (the allocation can be almost as cheap as allocating on the stack). If you can't avoid bo…
Yes, allocation can be incrementally be made cheaper with a copying GC. But you have to realize that it still comes at a cost, and allocation speed isn't always a bottleneck. You have a runtime process intercepting your code state, scanning and analyzing your heap and making a decision to keep, copy, or free. For anything larger than a trivially small object graph, that additional processing can easily overwhelm any…
Most generational gc's don't call free (the ones in Java can, but not per object, the reason for doing it is for returning memory to the OS). The GC will scan every _living_ object, but only those it is interested in. So if the decision to be made is which newly allocated objects should be copied over to the old gen, the gc will avoid scanning objects known to already be old.
The sentence you wrote made it seem like it scans the entire heap and decides which object should be kept, which should be copied and which should be free'd, but this is far from the case.
Since allocation with a GC is usually much cheaper than in non-GC systems (bump allocator) and since free is rarely, if ever, called, you can in theory get better performance from a GC than in a system with manual memory management. Of course, it all depends on the system.
The systems I write (mostly CRUD web apps) benefit from it, because requests to my service allocate little and run pretty fast, they're unlikely to survive to the next GC, and so I mostly pay for bump-allocation (which is cheap).
However, while the performance might be better my systems are prone to unexpected pauses while the GC does run. That pause might be less than the time spent in free in a manual memory managed system, but of course saving up all that time and spending it in one place does make it more noticable.
Thankfully, Java's G1 and collectors have managed to reduce these pauses so they're no longer noticable for my use. Pauses are less than 100ms (usually much lower) which is about the same variation I get from calling a third-party http service.
Earlier quoted context omitted.
> in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application Today you make make a very direct empirical comparison to see this - using the Graal compiler. This lets you compile exactly the same Java code either ahead-of-time or just-in-time, but using the same compiler logic except for the runtime information available when running jus…
>> Interestingly in 30 years I have not once heard of a case where this theoretical benefit has manifested as a clear advantage in any real world application when looking at the system as a whole. > The just-in-time code is (ignoring startup and warmup time) in my experience always faster, due to the extra runtime information. I don't think that result is surprising. The issue is that in the real world you can't igno…
Because JIT or AOT is just one little piece of the overall puzzle.
Earlier quoted context omitted.
Think of an use case where you have an API whose mode behavior is to do nothing, waiting for a call... but commonly gets calls that are very compute intense for short bursts of time. With the most common type of JITs, which profile once and compile once, I'm going to get code that is optimized for startup and initialization. If I have an "advanced" JIT, which is constantly deoptimizing and reoptimizing for whatever i…
The most commonly used JIT in the world is almost certainly the one in the OpenJDK, which is of the advanced kind. And it does not suffer from the problem you are talking about, because it only ever looks at code that is getting executed. Basically, it will do something like: interpret a function for a the first few thousand times it is executed, collecting execution metrics. After a threshold is reached, next time t…
OpenJ9 and ART also have similar capabilities.
Earlier quoted context omitted.
> And whichever way you look at it, the GC doesn't really have to outperform manual memory management to be extremely useful. It just has to be close enough, the correctness guarantees alone make it worth it as long as the performance difference is not too large, in many domains, not to mention the productivity boost and program readability. I agree with all of this. But it is a very different argument than the theor…
But Rust is designed to be blazing fast in many aspects, even if it uses a traditional malloc/free. Most likely if you compare a Rust and Java program that both run a loop creating a million node linked list and then starting over, you'll see that the Java program is much faster. I'll actually try to do that and confirm, but I believe it should be the case. Even if you were to allocate arrays (Vec) you may see the sa…
Tl;dr: JIT is a compiler that optimuzes certain parts of code after interpreter see it as hot (e.g. was executed many time). Thank to runtime information it can occasionally exceed performance of statically compiled language.
I don’t believe JITs can ever beat AOT compilation. However, if it can achieve parity in the execution of the critical path, that is sufficient to call it a win.
- Dynamic method calls that get de-virtualized, because there is only one visible implementation, including across shared libraries
- Inlining across shared libraries
- Adaptation of new CPU opcodes, for example a Java application written in 2000 could in theory make use of AVX, without being recompiled.
- In the same vein, that is how shading languages work
Earlier quoted context omitted.
I don’t believe JITs can ever beat AOT compilation. However, if it can achieve parity in the execution of the critical path, that is sufficient to call it a win.
> I don’t believe JITs can ever beat AOT compilation. In certain cases [1] JIT can beat AOT by a decent margin because it has access to live information that isn't available at compile time. In theory you could AOT compile, profile, and recompile but in practice my understanding is that nothing beats doing it live. Consider that if the workload suddenly changes in production and your heuristic violates an invariant a…
I just realized that the link color changes every time I load the page–I clicked on the article again and it was different from the last time I looked at it!
picking colours is really hard so I decided not to