31–40 of 84 posts
... and 1-2% improvement in performance via the register allocator.
If google has 10 million servers then a 2% improvement would be like freeing up 200,000 servers. That's a significant amount of value added!
Would be interesting to compare with gcc, which has a better implementation of both inlining and register allocation.
I'm very familiar with both inlining and register allocation in GCC (I even helped write one of the register allocation rewrite attempts, until vlad went even further down the rabbit hole of writing register allocators than we did)
RA was itself is historically more advanced in amount of optimization it does itself in GCC, but was still mostly a wash, because it's still much easier to work with LLVM's form and better optimizers were written.
LLVM also nowadays has a fairly advanced register allocator if you want it (PBQP) - it produces near optimal register allocations. It's not used though. It hasn't been worth it.
Inlining, they are not usefully comparable - they are very different approaches, both are good, and during the last inliner rewrite in LLVM, large amounts of comparisons were done a large number of times, and there isn't anything to suggest that GCC's inliner is really better (unlike RA, where the default GCC algorithm is certainly more advanced than the default LLVM RA, despite this being a deliberate decision).
We spent a lot of time optimizing inlining and RA/etc code when we transitioned to LLVM at Google (many years ago now)
At when we made that transition it was a net positive thing for fleet, and it was on a very large codebase.
The truth about compilers is there are no silver bullets. Things get faster mostly through careful tuning (which ML will hopefully replace), and much more rarely due to better algorithms.
As a result, i would expect GCC to not do any better here than LLVM.
I remember listening to one of the Lex Fridman interviews with Jim Keller where he said that modern branch prediction in CPUs was now done by “neural nets” in silicon. Does anyone here have any insight into this?
Would be interesting to compare with gcc, which has a better implementation of both inlining and register allocation.
I still want to see the estimated distance heuristic ( https://dspace.library.uvic.ca/bitstream/handle/1828/7107/Bu... ) in a modern compiler.
Earlier quoted context omitted.
What do you mean about GCC shooting itself in the head? What do you mean about its license?
GCC was for many years designed to be hard to work on/with => LLVM steals it's lunch GCC is under the GPL v3 . This does not bother me so much combined with the former it's bad for getting corporations to spend money on you
Earlier quoted context omitted.
today, learned heuristics have a couple of pitfalls that make them hard to add to such systems 1. they are usually hard to run efficiently 2. they are usually hard to explain The former is definitely changing with low precision formats like fp16 and useful coprocessors that can do matrix multiplications efficiently (M1, Intel). The latter hasn't been developed much and unless you're just training a model to memorize…
From my work, I've found it's really simple and straightforward to apply. 1. Choose a parameter for your compiler, xxx. 2. Have your ML model "choose compiler config parameter yyy." After the ML model "chooses" the config parameters, work backwards. 3. Determine why yyy is a better config parameter than xxx. It might not be! This system works, brilliantly. Cyborg intelligence, a combination of the human being and the…
Most interesting cases don't really look like this. The heuristic is applied to the user's code; it's not a one-time knob in the compiler. If it were, then you would likely be able to afford an exhaustive search to pick it & wouldn't need ml.
Compilers have soooo many heuristics. And a lot of it looks like a chess or Go game: You have a list of 100s of AST optimization passes (possible moves) that preserve the semantics of the program but you have limited compute with which to run iterations of these passes. What order and combination of these should you use?
> Better code optimization can significantly reduce the operational cost of large datacenter applications
They’re aiming to spend a bit more time compiling models, to reduce the scaled operational costs moving forward.
Earlier quoted context omitted.
GCC was for many years designed to be hard to work on/with => LLVM steals it's lunch GCC is under the GPL v3 . This does not bother me so much combined with the former it's bad for getting corporations to spend money on you
Stallman hasn't been relevant to GCC in 10-15 years. If people aren't doing research projects with it, it's because they don't like the code style (which is still weird but not monolithic) or their funders aren't using it in production.