Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

81–90 of 119 posts

Re: Large Language Models for Compiler Optimization

#81

Post by CJefferson »

wrsh07 wrote: post #65
CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

Are you compiling the code or interpreting?

Compiling C++ code, which means there is a chance you could prove transformations correct, but I’m not going to try it myself :)

Re: Large Language Models for Compiler Optimization

#82

Post by CJefferson »

darksaints wrote: post #66
CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

I can't imagine there would be any success with an LLM that isn't already known about via Satisfiability Modulo Theory.

I use SMT, but I can’t imagine how it would suggest good names for variables?

Re: Large Language Models for Compiler Optimization

#83

Post by PhilipRoman »

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order.

Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages are too low level for optimization. ML has the potential to extract back this lost information. Basically the opposite of lowering an IR.

Re: Large Language Models for Compiler Optimization

#84

Post by boomanaiden154 »

uoaei wrote: post #78
boomanaiden154 wrote: post #26

One of the biggest things that seems to be holding back ML in compilers right now is dataset size. This model was only trained on a gigabyte of source code, 30+% of that synthetic. Even on much simpler models, there have been massive performance gains by just throwing more data at them. Some experimentation with the original MLGO inlining model on a much bigger data corpus doubled the code-size wins. LLMs have also b…

What's holding them back is provable correctness. It's possible, nay, mandatory to constrain the outputs of the model at each step of generation in order to guarantee that a given structure or grammar is adhered to. If you can fine-tune the model with these constraints in place you can offload a lot of the effort that the LLM otherwise has to perform in comprehending correctness so it has more capacity for generating…

Most of the work in this space is not focused on neural compilation (having a ML model perform the transformation/entire compilation), but on replacing heuristics or phase ordering, where the issue of correctness falls back onto the compiler. For pretty much exactly the reasons you mentioned, neural compilation isn't really tractable.

This specific paper focuses on phase ordering, which should guarantee correctness, assuming the underlying transformations are correct. They do train the model to perform compilation, but as an auxiliary task.

Re: Large Language Models for Compiler Optimization

#85

Post by boomanaiden154 »

neonsunset wrote: post #79

I was hoping to see a more ambitious language model based project. Rather than just picking compiler arguments, an LLM could parse, and then transform AST into its more optimized form in a way that does not rely on a very formal way it is treated by compilers of today, apply guesstimate-based branch reordering and inlining heuristics not dissimilar to how a programmer would do so manually. Once done, a compiler could…

What's the benefit of having an LLM do those things in a way that guesstimates? There are big wins to be had in code-size and some wins to be had in performance related to inlining [1][2], but I think the implementation in the references directly tied into the compiler's inlining heuristic is a much better way to do that as it guarantees correctness. In addition, there's a reason that compilers basically ignore the `inline` keyword these days.

For branch reordering, techniques like BOLT [5] are pretty effectively able to reorder code layout at the binary level for big performance gains by using profile information. ML models can sometimes synthesize that information [3], but if I recall correctly, the performance of those models wasn't as good.

Neural compilation (like what you're describing) has been tried with LLMs [4], but has a lot of correctness problems currently, and I don't think it's going to be feasible anytime soon to do reinforcement learning for performance/code-size improvements.

1. https://arxiv.org/abs/2101.04808 2. https://arxiv.org/abs/2207.08389 3. https://arxiv.org/abs/2112.14679 4. https://ieeexplore.ieee.org/document/9926313 5. https://arxiv.org/abs/1807.06735

Re: Large Language Models for Compiler Optimization

#86

Post by boomanaiden154 »

PhilipRoman wrote: post #83

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order. Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages…

ML for phase ordering is just one problem that ML could solve within compilers.

Heuristic replacement (like loop unrolling) is another big one. For the specific case of loop unrolling, I would think lower level elements like how much iCache pressure the unrolling creates/whether or not the loop could fit in the DSB buffer would matter more.

For your point about existing IRs being too low-level, there has been a large push to try and work on that. MLIR has been used pretty extensively for that problem in ML applications, and languages like Rust have multiple higher level IRs. There's also a preliminary implementation of a Clang-IR for C/C++, and there's even be some work on higher level representations within LLVM-IR itself.

Re: Large Language Models for Compiler Optimization

#87

Post by kaba0 »

jebarker wrote: post #11
kukkamario wrote: post #7

Earlier quoted context omitted.

They are not using LLM to directly produce the result code, but as tool that lists which optimisations should be done and in which order, which is fairly complex problem to solve. But if optimisation passes are implemented correctly (which is anyway required for a functioning optimising compiler), it cannot produce incorrect code, maybe only suboptimal compared to default heuristics used.

If there's a list of known optimizations that preserve correctness then it becomes an optimization problem based on output length (as a proxy for cycle count). So is the idea that an LLM is more efficient than a search or direct optimization?

Cycle count is not a proper benchmark for performance on “modern” processors.

Re: Large Language Models for Compiler Optimization

#88

Post by kaba0 »

naveen99 wrote: post #29

Chatgpt4 can do source to source optimization which is pretty cool. I got it to beat gcc at -03 on simple small toy problems. and it can do similar things with python also. But it threw its hands up when I gave it a longer piece of code to optimize.

I don’t think any of those examples would be materially different than finding a related problem on stackoverflow and applying the answer.

LLMs are just good at searching and transforming between representations, but they really are not good at logical inferences.

Re: Large Language Models for Compiler Optimization

#89

Post by kaba0 »

quadrature wrote: post #33

I wonder if this requires a 7B parameter model.Assembly has a small grammar and is very constrained compared to natural language.

I don’t see why would the output language’s complexity matter - that’s clearly not the hard part. You need plenty of “thinking” to do for outputting sensible assembly, let alone whole programs.

With that said, it is not doing neural compilation as others have mentioned, it’s only about ordering/enabling different phases of the compiler based on ML, over the current, simpler heuristics.

Re: Large Language Models for Compiler Optimization

#90

Post by smusamashah »

Do LLMs need to see some information just once to answer about it? My understanding was that they need to see many examples of the same thing to be able to answer it correctly (generate text around it correctly).

Shouldn't this mean that it is safe to ask GPTs about something proprietary if it was just once because rare examples should just disappear in weights of everything else. And this also means that even GPT4 won't be able to answer any queries about obscure or rare knowledge it has seen in its training dataset.

Post reply on HN