Google ML Compiler Inlining Achieves 3-7% Reduction in Size

... and 1-2% improvement in performance via the register allocator.

They trained (and report) two optimization strategies:

- inline-for-size

  We trained the inlining-for-size policy on a large internal software package containing 30k modules. The trained policy is generalizable when applied to compile other software and achieves a 3% ~ 7% size reduction.

- regalloc

  with 0.3% ~1.5% improvements in queries per second (QPS) on a set of internal large-scale datacenter applications

  Try it Yourself
  Check out the open-sourced end-to-end data collection and training solution on github and a demo that uses policy gradient to train an inlining-for-size policy.

  https://github.com/google/ml-compiler-opt

  https://github.com/google/ml-compiler-opt/blob/main/docs/demo/demo.md

With code, that's awesome—what I like to see.

Would be interesting to compare with gcc, which has a better implementation of both inlining and register allocation.

Neat.

It seems like it's sort of built in to LLVM right now, but not usable unless you build LLVM yourself with pretrained models as part of the build process?

Compilers have soooo many heuristics. And a lot of it looks like a chess or Go game: You have a list of 100s of AST optimization passes (possible moves) that preserve the semantics of the program but you have limited compute with which to run iterations of these passes. What order and combination of these should you use?