Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

71–80 of 119 posts

Re: Large Language Models for Compiler Optimization

#71

Post by aureianimus »

CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

Not strictly what you're looking for, but in Lean (functional language/theorem prover), there's some interesting work being done. Using the tool actually shows which suggestions will compile, which certifies correctness to some degree. https://leandojo.org/

Re: Large Language Models for Compiler Optimization

#72

Post by BruceEel »

boomanaiden154 wrote: post #25
drug-freedom wrote: post #22

Earlier quoted context omitted.

One example where a LLM might be better is which functions to inline. Current compilers use a complex set of heuristics, a more holistic approach the kind neural networks do might outperform.

For function inlining specifically, I'm not sure LLMs are necessarily the right choice. The original MLGO paper [1] demonstrated a big code-size improvement with a ML model for making inlining decisions (7-20% code size wins), but they used tens of engineered features. Maybe a LLM could squeeze some additional size wins out, but maybe not [2]. Additionally, there are other factors to consider when productionizing the…

I'm probably being very naive with this but, could mechanistic interpretability play a role here? Specifically, to experimentally short-list the LLM's most effective optimizations, then try to peek inside the LLM and perhaps fish out some novel optimization algorithm that could be efficiently implemented without the LLM?

Re: Large Language Models for Compiler Optimization

#73

Post by alex_lav »

cypress66 wrote: post #68
alex_lav wrote: post #64

Earlier quoted context omitted.

I work in a research facility, so the biggest fear is that our super top secret elite info will leak. The reality is it'd be used to refactor a lot of terrible code.

Use local models if you don't want to send your data to OpenAI

> We have a total ban on AI for source code analysis

Re: Large Language Models for Compiler Optimization

#74

Post by boomanaiden154 »

BruceEel wrote: post #72
boomanaiden154 wrote: post #25

Earlier quoted context omitted.

For function inlining specifically, I'm not sure LLMs are necessarily the right choice. The original MLGO paper [1] demonstrated a big code-size improvement with a ML model for making inlining decisions (7-20% code size wins), but they used tens of engineered features. Maybe a LLM could squeeze some additional size wins out, but maybe not [2]. Additionally, there are other factors to consider when productionizing the…

I'm probably being very naive with this but, could mechanistic interpretability play a role here? Specifically, to experimentally short-list the LLM's most effective optimizations, then try to peek inside the LLM and perhaps fish out some novel optimization algorithm that could be efficiently implemented without the LLM?

I did a bit of work on this last summer on (much) smaller models [1] and it was briefly discussed towards the end of last year's MLGO panel [2]. For heuristic replacements specifically, you might be able to glean some things (or just use interpretable models like decision trees), but something like a neural network works fundamentally differently than the existing heuristics, so you probably wouldn't see most of the performance gains. For just tuning heuristics, the usual practice is to make most of the parameters configurable and then use something like bayesian optimization to try and find an optimal set, and this is sometimes done as a baseline in pieces of ML-in-compiler research.

1. https://github.com/google/ml-compiler-opt/pull/109 2. https://youtu.be/0uUKDQyn1Z4?si=PHrx9RICJIiA3E6C

Re: Large Language Models for Compiler Optimization

#75

Post by isaacfung »

boomanaiden154 wrote: post #26

One of the biggest things that seems to be holding back ML in compilers right now is dataset size. This model was only trained on a gigabyte of source code, 30+% of that synthetic. Even on much simpler models, there have been massive performance gains by just throwing more data at them. Some experimentation with the original MLGO inlining model on a much bigger data corpus doubled the code-size wins. LLMs have also b…

Quality matters just as much as quantity.

LIMA: Less Is More for Alignment https://arxiv.org/abs/2305.11206

AlpaGasus: Training A Better Alpaca with Fewer Data https://arxiv.org/abs/2307.08701

Textbooks Are All You Need II: phi-1.5 technical report https://arxiv.org/abs/2309.05463

Re: Large Language Models for Compiler Optimization

#76

Post by BruceEel »

boomanaiden154 wrote: post #74
BruceEel wrote: post #72

Earlier quoted context omitted.

I'm probably being very naive with this but, could mechanistic interpretability play a role here? Specifically, to experimentally short-list the LLM's most effective optimizations, then try to peek inside the LLM and perhaps fish out some novel optimization algorithm that could be efficiently implemented without the LLM?

I did a bit of work on this last summer on (much) smaller models [1] and it was briefly discussed towards the end of last year's MLGO panel [2]. For heuristic replacements specifically, you might be able to glean some things (or just use interpretable models like decision trees), but something like a neural network works fundamentally differently than the existing heuristics, so you probably wouldn't see most of the…

Ah interesting. Hadn't seen that presentation, thanks for sharing!

Re: Large Language Models for Compiler Optimization

#77

Post by isaacfung »

aureianimus wrote: post #71
CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

Not strictly what you're looking for, but in Lean (functional language/theorem prover), there's some interesting work being done. Using the tool actually shows which suggestions will compile, which certifies correctness to some degree. https://leandojo.org/

Similar ideas

Certified Reasoning with Language Models https://github.com/gpoesia/certified-reasoning

It's based on Peano, a theorem proving environment Peano: Learning Formal Mathematical Reasoning https://arxiv.org/abs/2211.15864

(https://github.com/kyegomez/LOGICGUIDE claims to implement the same paper as the first repo but it is fake)

Re: Large Language Models for Compiler Optimization

#78

Post by uoaei »

boomanaiden154 wrote: post #26

One of the biggest things that seems to be holding back ML in compilers right now is dataset size. This model was only trained on a gigabyte of source code, 30+% of that synthetic. Even on much simpler models, there have been massive performance gains by just throwing more data at them. Some experimentation with the original MLGO inlining model on a much bigger data corpus doubled the code-size wins. LLMs have also b…

What's holding them back is provable correctness.

It's possible, nay, mandatory to constrain the outputs of the model at each step of generation in order to guarantee that a given structure or grammar is adhered to. If you can fine-tune the model with these constraints in place you can offload a lot of the effort that the LLM otherwise has to perform in comprehending correctness so it has more capacity for generating good content. To be sure, quality and quantity of data are important, but it's all too easy to introduce subtle bugs that take years to tease out if you don't adhere to the right constraints.

Re: Large Language Models for Compiler Optimization

#79

Post by neonsunset »

I was hoping to see a more ambitious language model based project.

Rather than just picking compiler arguments, an LLM could parse, and then transform AST into its more optimized form in a way that does not rely on a very formal way it is treated by compilers of today, apply guesstimate-based branch reordering and inlining heuristics not dissimilar to how a programmer would do so manually.

Once done, a compiler could perform final AST validation and either route it back to LLM to fix it or auto-fix most common cases.

Re: Large Language Models for Compiler Optimization

#80

Post by haltist »

neonsunset wrote: post #79

I was hoping to see a more ambitious language model based project. Rather than just picking compiler arguments, an LLM could parse, and then transform AST into its more optimized form in a way that does not rely on a very formal way it is treated by compilers of today, apply guesstimate-based branch reordering and inlining heuristics not dissimilar to how a programmer would do so manually. Once done, a compiler could…

How do you validate semantic equivalence?
Post reply on HN