I remember listening to one of the Lex Fridman interviews with Jim Keller where he said that modern branch prediction in CPUs was now done by “neural nets” in silicon. Does anyone here have any insight into this?
Google ML Compiler Inlining Achieves 3-7% Reduction in Size
81–84 of 84 posts
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#82The (current) title used on HN ( “Google ML Compiler Inlining Achieves 3-7% Reduction in Size” ) is confusing. It made me think Google is using ML ( https://en.wikipedia.org/wiki/ML_(programming_language) ) It also is not the one of the referred page, which is “MLGO: A Machine Learning Framework for Compiler Optimization” . This is about an LLVM extension that uses Machine Learning. I think it would be better to chan…
There is also the much larger confusion arising of whether it's an ML compiler in the sense of being a specialized compiler for ML code (compiles a description of a network to efficient ML-specific low-level IR to enable further optimization and code generation), or whether it's an ML compiler, a compiler (or part of it) that uses ML.
The first meaning is extremly common, the top 2 results for "ML Compiler" on google returns[1][2], both using it in the first sense. It's not that ML or ML-like techniques in compiler writing is new but the first sense is definitely at least as popular an interpretation.
[1] https://huyenchip.com/2021/09/07/a-friendly-introduction-to-...
[2] https://petewarden.com/2021/12/24/why-are-ml-compilers-so-ha...
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#83Earlier quoted context omitted.
Looks like they do have a pretrained model: https://github.com/google/ml-compiler-opt/releases/tag/inlin... The code will by default auto-download it during the build process. It's about 800 kbytes, which seems very reasonable for something that will reduce the generated code size by gigabytes for a large codebase. Note that the open-source-ness is dubious... They say it was trained using an optimizer which isn't ope…
Yes - the default model really needs to be trained with an opensource optimizer on a corpus of open source code (ie. with a license at least as permissive as llvm itself). A blob trained with proprietary google tech on a proprietary google codebase isn't opensource. Even if it were, Google C++ differs in style quite widely from typical C++, so the model probably isn't as good as a model trained on all of github.
Someone created a github issue for this:
https://github.com/llvm/llvm-project/issues/56432
Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size
#84They trained (and report) two optimization strategies: - inline-for-size We trained the inlining-for-size policy on a large internal software package containing 30k modules. The trained policy is generalizable when applied to compile other software and achieves a 3% ~ 7% size reduction. - regalloc with 0.3% ~1.5% improvements in queries per second (QPS) on a set of internal large-scale datacenter applications Try it…
For those not familiar with the space, how significant an impact is this, particularly at Google scale?