Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google ML Compiler Inlining Achieves 3-7% Reduction in Size

ai.googleblog.com

Post reply on HN
Print view View original

51–60 of 84 posts

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#51

Post by londons_explore »

jamesfinlayson wrote: post #46
londons_explore wrote: post #44

Earlier quoted context omitted.

How big are the models? If just a few megabytes, why not just check them into the llvm git repo?

I'm not sure - https://github.com/google/ml-compiler-opt#pretrained-models says the they sometimes get released on GitHub but I couldn't see anything obvious in that repo.

Looks like they do have a pretrained model:

https://github.com/google/ml-compiler-opt/releases/tag/inlin...

The code will by default auto-download it during the build process. It's about 800 kbytes, which seems very reasonable for something that will reduce the generated code size by gigabytes for a large codebase.

Note that the open-source-ness is dubious... They say it was trained using an optimizer which isn't opensource, and that the results are significantly better ('more generalizable') than using the open source one. In my view, if the code that makes a binary blob isn't opensource, then the project isn't opensource...

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#53

Post by NavinF »

pabs3 wrote: post #42
londons_explore wrote: post #41

Earlier quoted context omitted.

It'll be 100% deterministic. Reproducibility is a big part of Google's internal build system, and they wouldn't be able to deploy something that broke that.

What about reproducibility of the compiler binaries? I got the impression that model training itself isn't deterministic across training hardware? Since the model is embedded in the binaries, then the binaries aren't really reproducible if the model isn't. How big is the data used to train the model? How costly is it to train the model?

It’s pretty easy to train models deterministically. Just use the same batch size, float type, initial seed, and flip this flag: https://www.tensorflow.org/api_docs/python/tf/config/experim...

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#55

Post by NavinF »

localhost wrote: post #32

I remember listening to one of the Lex Fridman interviews with Jim Keller where he said that modern branch prediction in CPUs was now done by “neural nets” in silicon. Does anyone here have any insight into this?

If you search “AMD Zen neural branch prediction” you’ll find several references to this, but very few details. I dunno if perceptrons really count as “neural nets” these days since it’s just a weighted average. Lots of computations are technically neural nets if we use such broad definitions.

It’d be pretty cool to see an x86 language model in future CPUs. I have no doubt that compute will continue to scale faster than memory access and the relative cost of pipeline stalls will never go down.

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#56

Post by londons_explore »

londons_explore wrote: post #51
jamesfinlayson wrote: post #46

Earlier quoted context omitted.

I'm not sure - https://github.com/google/ml-compiler-opt#pretrained-models says the they sometimes get released on GitHub but I couldn't see anything obvious in that repo.

Looks like they do have a pretrained model: https://github.com/google/ml-compiler-opt/releases/tag/inlin... The code will by default auto-download it during the build process. It's about 800 kbytes, which seems very reasonable for something that will reduce the generated code size by gigabytes for a large codebase. Note that the open-source-ness is dubious... They say it was trained using an optimizer which isn't ope…

Yes - the default model really needs to be trained with an opensource optimizer on a corpus of open source code (ie. with a license at least as permissive as llvm itself).

A blob trained with proprietary google tech on a proprietary google codebase isn't opensource. Even if it were, Google C++ differs in style quite widely from typical C++, so the model probably isn't as good as a model trained on all of github.

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#57

Post by xdfgh1112 »

zasdffaa wrote: post #50
tadeegan wrote: post #10

Compilers have soooo many heuristics. And a lot of it looks like a chess or Go game: You have a list of 100s of AST optimization passes (possible moves) that preserve the semantics of the program but you have limited compute with which to run iterations of these passes. What order and combination of these should you use?

Which you should use are often unknowable because it's often about how the code is driven by the data - you need to know the characteristics of the data to know how the code will behave, only then can you meaningfully talk about picking a 'better' heuristic. Analogy: otherwise you're just optimising the design of a car. But optimising it for what? speed, efficiency, reliability, price, weight, carrying capacity... Yo…

Profile guided optimisation is a thing.

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#58

Post by zasdffaa »

xdfgh1112 wrote: post #57
zasdffaa wrote: post #50

Earlier quoted context omitted.

Which you should use are often unknowable because it's often about how the code is driven by the data - you need to know the characteristics of the data to know how the code will behave, only then can you meaningfully talk about picking a 'better' heuristic. Analogy: otherwise you're just optimising the design of a car. But optimising it for what? speed, efficiency, reliability, price, weight, carrying capacity... Yo…

Profile guided optimisation is a thing.

You are quite right, I should have stated that.

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#59

Post by davidatbu »

Someone wrote: post #9

The (current) title used on HN ( “Google ML Compiler Inlining Achieves 3-7% Reduction in Size” ) is confusing. It made me think Google is using ML ( https://en.wikipedia.org/wiki/ML_(programming_language) ) It also is not the one of the referred page, which is “MLGO: A Machine Learning Framework for Compiler Optimization” . This is about an LLVM extension that uses Machine Learning. I think it would be better to chan…

100% agreed that the actual title is much better than the editorialized title.

Re: Google ML Compiler Inlining Achieves 3-7% Reduction in Size

#60

Post by rfoo »

NavinF wrote: post #53
pabs3 wrote: post #42

Earlier quoted context omitted.

What about reproducibility of the compiler binaries? I got the impression that model training itself isn't deterministic across training hardware? Since the model is embedded in the binaries, then the binaries aren't really reproducible if the model isn't. How big is the data used to train the model? How costly is it to train the model?

It’s pretty easy to train models deterministically. Just use the same batch size, float type, initial seed, and flip this flag: https://www.tensorflow.org/api_docs/python/tf/config/experim...

Caveat:

> on the same hardware

Post reply on HN