I wonder how tied the heuristic is to a particular compiler IR. E.g. can the decision procedure be boiled down to a form that could be plugged into other compilers that need an inlining heuristic?
Google ML Compiler Inlining Achieves 3-7% Reduction in Size
71–80 of 84 posts
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#72Is this the start of a future where we can write high level code (Idris, Agda, Coq) and the resulting code will run as fast (and as safe) as RUST? Interesting.
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#73Compilers have soooo many heuristics. And a lot of it looks like a chess or Go game: You have a list of 100s of AST optimization passes (possible moves) that preserve the semantics of the program but you have limited compute with which to run iterations of these passes. What order and combination of these should you use?
This is known as the "phase ordering problem". Despite the super-exponential (even unbounded, because phases can be repeated) space, I think this is more tractable for certain subsets. For example, in sea of nodes compilers, like HotSpot, Graal, and TurboFan, forward dataflow analyses that do reductions (i.e. always make the graph smaller and/or cheaper) can be combined into one mega-pass where they positively interact each other. The problem is that some problems are not forward data flow problems and some transformations are not simple reductions. Also, some phases make use of type information which can be destroyed or obscured by lowering.
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#74Using ML inside compilers has a lot of untapped potential I think. People think of a compiler as an AI when they're actually very stupid in terms of the number of decisions available to them. Feedback is the lifeblood of intelligent performance, it is more than possible to fake that feedback using AI. E.g. Your error callback is on the balance of probability going to be called less than the (say) core matrix multiply…
> when they're actually very stupid in terms of the number of decisions available to them.
I don't think this is a fair characterization, though I agree overall that ML has a lot of potential in compiler optimization.
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#75Earlier quoted context omitted.
Variants of this have been tried. What are you hoping for?
> Variants of this have been tried Have a link?
Honestly, no, because it's been decades at this point.
While not as formally written down, this is the most common way that profiling info was ever used in RA.
Using both the execution frequency and use distance to calculate which registers should be spilled and where to spill them was done even in toy research compilers when I was heavy into this part of the world.
I remember seeing implementations in IBM's XLC, at least two of the RA's we implemented in GCC did this, etc.
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#76As someone who doesn't follow compilers too much, is 3-7% considered good? Throwing a whole ML model in to the compiler seems like a pretty large boost to complexity for a small gain in performance.
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#77I wonder how well it performs on Rust code, which is probably different in terms of patterns from C++ code. They mention the Fuchsia project but they seem to have focused on its C++ components. There is actually more Rust inside Fuchsia now than C++ [0]. I also wonder how this would look like for mainlining. Should the LLVM project depend on tensorflow now? IIRC tensorflow itself depends on LLVM so to avoid circular…
>Should the LLVM project depend on tensorflow now?
This bit from the article seem to be relevant.
"The TensorFlow model is embedded with XLA AOT, which converts the model into executable code. This avoids TensorFlow runtime dependency"
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#78Isn't this always going to be less efficient than profile guided optimization? Of course that assumes you can profile your program with realistic "production" workloads, and it'll need two compilation passes, so having sensible ML defaults boxes sense. But it's odd not to compare this to PGO in terms of the resulting performance.
It's not odd because the result is on top of PGO. You would never adopt this approach if you did not already have PGO. The gains from PGO will be much larger.
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#79As someone who doesn't follow compilers too much, is 3-7% considered good? Throwing a whole ML model in to the compiler seems like a pretty large boost to complexity for a small gain in performance.
If you can fit 5% more functionality into a fixed size that is nice. Especially if you are only using this optimization level for the final production build that needs to be squeezed onto a small (and therefore cheaper) ROM.
Although I do agree that these days most projects will have media that takes up the majority of the space.
The stat that seems clearly impressive for me is that their register allocator claims "0.3% ~1.5% improvements in queries per second" which is a huge cost savings for operations at the scale of Google. If you have 100 datacenters of software running you can conceivably turn one of them off. (Or more likely slow down future building and expansion plans). Of course for most people compute costs aren't a significant expense.