41–50 of 84 posts
I wonder if the ML is deterministic or not. Otherwise you compile twice and get completely different binaries.
Reproducibility is a big part of Google's internal build system, and they wouldn't be able to deploy something that broke that.
I wonder if the ML is deterministic or not. Otherwise you compile twice and get completely different binaries.
It'll be 100% deterministic. Reproducibility is a big part of Google's internal build system, and they wouldn't be able to deploy something that broke that.
Neat. It seems like it's sort of built in to LLVM right now, but not usable unless you build LLVM yourself with pretrained models as part of the build process?
Using ML inside compilers has a lot of untapped potential I think. People think of a compiler as an AI when they're actually very stupid in terms of the number of decisions available to them. Feedback is the lifeblood of intelligent performance, it is more than possible to fake that feedback using AI. E.g. Your error callback is on the balance of probability going to be called less than the (say) core matrix multiply…
I also played around with using ml to optimise auto scaling of CI instances. (taking time of day and queue sizes into account)
Neat. It seems like it's sort of built in to LLVM right now, but not usable unless you build LLVM yourself with pretrained models as part of the build process?
How big are the models? If just a few megabytes, why not just check them into the llvm git repo?
Earlier quoted context omitted.
It'll be 100% deterministic. Reproducibility is a big part of Google's internal build system, and they wouldn't be able to deploy something that broke that.
What about reproducibility of the compiler binaries? I got the impression that model training itself isn't deterministic across training hardware? Since the model is embedded in the binaries, then the binaries aren't really reproducible if the model isn't. How big is the data used to train the model? How costly is it to train the model?
Earlier quoted context omitted.
It'll be 100% deterministic. Reproducibility is a big part of Google's internal build system, and they wouldn't be able to deploy something that broke that.
What about reproducibility of the compiler binaries? I got the impression that model training itself isn't deterministic across training hardware? Since the model is embedded in the binaries, then the binaries aren't really reproducible if the model isn't. How big is the data used to train the model? How costly is it to train the model?
> The TensorFlow model is embedded with XLA AOT, which converts the model into executable code.
Taking the TF model to executable code should be deterministic.
Generating the TF model might not be (depends on implementation and hardware).
But it's unclear from this if this is a problem in practice - it's not uncommon for non-deterministic models to end up producing the same output because you perform thresholding/quantizing or some analogous process to convert it into a classification style output.
I.E. here, you are generating operations, which you can setup as a classification problem: Given this input and this history what is the next operation.
And of course you can always got back to your saved model and generate the code from that.
While the exact scores for the next operation might be non-deterministic you always end up with the highest scoring one being the same.
Compilers have soooo many heuristics. And a lot of it looks like a chess or Go game: You have a list of 100s of AST optimization passes (possible moves) that preserve the semantics of the program but you have limited compute with which to run iterations of these passes. What order and combination of these should you use?
Analogy: otherwise you're just optimising the design of a car. But optimising it for what? speed, efficiency, reliability, price, weight, carrying capacity... You first need to know how it's expected to be used.
I guess local inlining might sometimes be an unconditional win, but even then only under specific circumstances.
(disclaimer: I know something but am not an expert)