Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

111–119 of 119 posts

Re: Large Language Models for Compiler Optimization

#111

Post by cec »

Lynix48 wrote: post #95

I was thinking about writing my bachelor's thesis about LLMs and compilers. My idea is to maybe attempt to write a C (to assembler)-Compiler and see how well it does. Attempting to convert assembler to C using an LLM would be interesting too, but the results would probably be poor since there is just so much information that gets lost when compiling, so the C code would be pretty statistical. I guess I could improve…

There’s a researcher at Edinburgh who is looking into this. You may want to reach out. Check the related work section in this paper

Re: Large Language Models for Compiler Optimization

#112

Post by cec »

quadrature wrote: post #105
kaba0 wrote: post #89

Earlier quoted context omitted.

I don’t see why would the output language’s complexity matter - that’s clearly not the hard part. You need plenty of “thinking” to do for outputting sensible assembly, let alone whole programs. With that said, it is not doing neural compilation as others have mentioned, it’s only about ordering/enabling different phases of the compiler based on ML, over the current, simpler heuristics.

Exactly, since its not doing whole program synthesis im thinking it could be done with fewer parameters. However program synthesis is part of the loss function.

Program synthesis is part of the loss function, which is what makes it a auxiliary learning task.

We haven’t experimented with model size yet, we just used the same configuration as the smallest Code Llama. We did play with dataset size and found thah performance tracks the usual scaling laws. Details in the paper

Re: Large Language Models for Compiler Optimization

#113

Post by cec »

kolbe wrote: post #31

> Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler, outperforming two state-of-the-art baselines that require thousands of compilations. If that's their target, then what's the point? LLVM's optimizations are not done to minimize instructions but to maximize performance. On modern processors, these can be very different things.

Hey, we’re targeting code size in this work, not runtime performance. You would use an option like -O3 to optimize for runtime and -Oz to optimize for code size. The pass pipelines are different for both

Re: Large Language Models for Compiler Optimization

#114

Post by cec »

primordialsoup wrote: post #51

This is very interesting work, but it's not really a LLM. It doesn't have language abilities. They should have called it a seq2seq model, but I think that term is not in vogue these days :)

We use the same architecture as other LLMs, but we include no natural language in our pretraining. We figured a single-domain training corpus would make evaluation easier. We’ll be looking at layering this on top of something like Code Llama next

Re: Large Language Models for Compiler Optimization

#115

Post by quadrature »

cec wrote: post #112
quadrature wrote: post #105

Earlier quoted context omitted.

Exactly, since its not doing whole program synthesis im thinking it could be done with fewer parameters. However program synthesis is part of the loss function.

Program synthesis is part of the loss function, which is what makes it a auxiliary learning task. We haven’t experimented with model size yet, we just used the same configuration as the smallest Code Llama. We did play with dataset size and found thah performance tracks the usual scaling laws. Details in the paper

Thanks for the reply!. This is really interesting research, hope to see more from your team.

Re: Large Language Models for Compiler Optimization

#116

Post by eru »

hoosieree wrote: post #102
eru wrote: post #93

Earlier quoted context omitted.

> You don't ask the model what instructions to emit. You could still do that, you'd just also need to ask the model for a proof. (But I guess that's much harder than heuristically picking which passes to apply.)

This is the right way to interact with LLMs in general. Ask for what you want, but independently verify the result. Don't be like that lawyer "...but I asked ChatGPT if it was telling the truth, and it said yes!"

What you describe is useful, but it's not really what my comment described.

My comment was describing the wacky idea of asking the model to come up with a formal, machine-checkable mathematical proof of correctness, too. That's hard in general.

The idea in the article of just letting the model pick between different, already proven-correct, optimization passes is much saner most of the time.

Re: Large Language Models for Compiler Optimization

#117

Post by misrasaurabh1 »

I see a lot of interest in this field which is amazing! I have been working on this problem for a while and have made a lot of progress! If you are interested in building a general purpose code performance optimizer, I am hiring founding engineers. We are still in stealth so can't say too much but we already have great results on customers are already funded by some of the best investors of silicon valley. If you might be interested, pls reach out at misra.saurabh1@gmail.com

Re: Large Language Models for Compiler Optimization

#118

Post by wrsh07 »

CJefferson wrote: post #81
wrsh07 wrote: post #65

Earlier quoted context omitted.

Are you compiling the code or interpreting?

Compiling C++ code, which means there is a chance you could prove transformations correct, but I’m not going to try it myself :)

At the very least transformations that produce identical assembly are safe. I think.

Re: Large Language Models for Compiler Optimization

#119

Post by parentheses »

What is interesting to me about this approach is that they're fine tuning an existing model. If they could train a model using input/output pairs from scratch and entirely on the optimization task, I could see a smaller LLM performing drastically better than the compiled output - according to whatever the loss function is implemented to optimize for.
Post reply on HN