Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

101–110 of 119 posts

Re: Large Language Models for Compiler Optimization

#101

Post by hoosieree »

JonChesterfield wrote: post #97

From the abstract: - 3.0% improvement in reducing instruction counts over the compiler - generating compilable code 91% of the time - perfectly emulating the output of the compiler 70% of the time. I read through the paper to see what perfectly emulating the output means. In this case, I think it's that it's also possible to get the same code out of the compiler using a different pass order. I was hoping for still pa…

I've just skimmed it but I think they're not using the model's output for codegen, but rather to choose what compiler passes to use and in what order.

Re: Large Language Models for Compiler Optimization

#102

Post by hoosieree »

eru wrote: post #93
PhilipRoman wrote: post #83

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order. Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages…

> You don't ask the model what instructions to emit. You could still do that, you'd just also need to ask the model for a proof. (But I guess that's much harder than heuristically picking which passes to apply.)

This is the right way to interact with LLMs in general. Ask for what you want, but independently verify the result. Don't be like that lawyer "...but I asked ChatGPT if it was telling the truth, and it said yes!"

Re: Large Language Models for Compiler Optimization

#104

Post by cypress66 »

jmgao wrote: post #70
cypress66 wrote: post #67

Earlier quoted context omitted.

Printf is horribly slow compared to puts

puts isn't applicable here because you actually need to format the %d, but any compiler worth using (i.e. not MSVC: https://gcc.godbolt.org/z/zx74vY1za ) will optimize a printf of a constant string ending in a newline into a puts.

No you don't need to format the %d. The same way you collapsed the loop into the constant 5, you collapse that printf into puts("5\n")

Re: Large Language Models for Compiler Optimization

#105

Post by quadrature »

kaba0 wrote: post #89
quadrature wrote: post #33

I wonder if this requires a 7B parameter model.Assembly has a small grammar and is very constrained compared to natural language.

I don’t see why would the output language’s complexity matter - that’s clearly not the hard part. You need plenty of “thinking” to do for outputting sensible assembly, let alone whole programs. With that said, it is not doing neural compilation as others have mentioned, it’s only about ordering/enabling different phases of the compiler based on ML, over the current, simpler heuristics.

Exactly, since its not doing whole program synthesis im thinking it could be done with fewer parameters. However program synthesis is part of the loss function.

Re: Large Language Models for Compiler Optimization

#106

Post by cec »

JonChesterfield wrote: post #97

From the abstract: - 3.0% improvement in reducing instruction counts over the compiler - generating compilable code 91% of the time - perfectly emulating the output of the compiler 70% of the time. I read through the paper to see what perfectly emulating the output means. In this case, I think it's that it's also possible to get the same code out of the compiler using a different pass order. I was hoping for still pa…

Hey, author here. We use machine learning to generate a list of _optimization passes_ for the compiler to run. These optimization passes give us a 3.0% improvement over the default (-Oz), and are what generates correct code. We don't do anything to the code that breaks semantics (assuming no bugs in the compiler passes; some nuance is needed here ;) ).

We also train the model to generate what it thinks the optimized code will look like. We find that this helps the model choose better pass lists, but obviously the code cannot be trusted and semantics are not guaranteed. It only compiles in 91% of cases. "Perfectly emulating the output of the compiler" means the model spat out code that is character-for-character identical to what the compiler generates with the given pass list (even choosing the same variable names etc). IMO this is no mean feat, but is still a long way to go before using LLMs for codegen. We provide a bunch of examples in the paper of things that LLMs can and cannot do.

Re: Large Language Models for Compiler Optimization

#108

Post by Yvan-xy »

cec wrote: post #106
JonChesterfield wrote: post #97

From the abstract: - 3.0% improvement in reducing instruction counts over the compiler - generating compilable code 91% of the time - perfectly emulating the output of the compiler 70% of the time. I read through the paper to see what perfectly emulating the output means. In this case, I think it's that it's also possible to get the same code out of the compiler using a different pass order. I was hoping for still pa…

Hey, author here. We use machine learning to generate a list of _optimization passes_ for the compiler to run. These optimization passes give us a 3.0% improvement over the default (-Oz), and are what generates correct code. We don't do anything to the code that breaks semantics (assuming no bugs in the compiler passes; some nuance is needed here ;) ). We also train the model to generate what it thinks the optimized…

Hey, I just read through this paper. The phase ordering issues are currently based on heuristics. I noticed that you're only using the instruction count of LLVM as the measurement. However, this metric might not accurately reflect the program's actual performance and code size. LLVM has instructions like GEP that can translate into several lines of assembly code. Additionally, I suggest trying to run some large benchmarks like SPEC to demonstrate the performance benefits of using the LLM.

Re: Large Language Models for Compiler Optimization

#109

Post by cec »

Yvan-xy wrote: post #108
cec wrote: post #106

Earlier quoted context omitted.

Hey, author here. We use machine learning to generate a list of _optimization passes_ for the compiler to run. These optimization passes give us a 3.0% improvement over the default (-Oz), and are what generates correct code. We don't do anything to the code that breaks semantics (assuming no bugs in the compiler passes; some nuance is needed here ;) ). We also train the model to generate what it thinks the optimized…

Hey, I just read through this paper. The phase ordering issues are currently based on heuristics. I noticed that you're only using the instruction count of LLVM as the measurement. However, this metric might not accurately reflect the program's actual performance and code size. LLVM has instructions like GEP that can translate into several lines of assembly code. Additionally, I suggest trying to run some large bench…

Hey, yes that's right, and good callout on GEP instructions. We admit that instruction count is a bit handwavy, but we use it as a starting point as that's what the prior works we compare against optimize for. We'll be looking at true binary size next, and code runtime after.

Re: Large Language Models for Compiler Optimization

#110

Post by cec »

JonChesterfield wrote: post #98
wyldfire wrote: post #5

> understanding. We evaluate on a large suite of test programs. Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler, 3% code size reduction is really good. The challenge will be having codegen like this that someone is willing to support. And for that they'd want to be able to reason about why the compiler made this decision or that one. IIUC that's an outstanding problem for AI…

3% code size reduction while changing semantics is borderline worthless*. 3% code size reduction without changing semantics would be more interesting but might still be a bad thing for performance. *fast-math etc is a thing, where similar-enough output is fine

Hey Jon, we don't change semantics. We just choose optimization passes to run to match a particular input code. Agreed that code size wins could regress performance. We don't measure that yet, but will be looking into it next. There are some applications where codesize above-all-else is what to optimize for.
Post reply on HN