Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

11–20 of 119 posts

Re: Large Language Models for Compiler Optimization

#11

Post by jebarker »

kukkamario wrote: post #7
jebarker wrote: post #3

Earlier quoted context omitted.

It seems like a poor fit to me precisely because correctness is boolean, difficult to measure and getting it wrong is very bad. I do think there's a place for AI here but it's probably not LLMs in their current form.

They are not using LLM to directly produce the result code, but as tool that lists which optimisations should be done and in which order, which is fairly complex problem to solve. But if optimisation passes are implemented correctly (which is anyway required for a functioning optimising compiler), it cannot produce incorrect code, maybe only suboptimal compared to default heuristics used.

If there's a list of known optimizations that preserve correctness then it becomes an optimization problem based on output length (as a proxy for cycle count). So is the idea that an LLM is more efficient than a search or direct optimization?

Re: Large Language Models for Compiler Optimization

#12

Post by mathisfun123 »

klohto wrote: post #9
CalChris wrote: post #4

There's a half day tutorial at the LLVM Developers Meeting on this, ML-Guided Compiler Optimization in LLVM . However, the authors of this paper aren't giving that tutorial.

MLGO uses RL

maybe today but MLGO, the initiative, uses whatever. To wit: chris cummins (first author here) was on last year's MLGO panel.

Re: Large Language Models for Compiler Optimization

#13

Post by TheLoafOfBread »

wyldfire wrote: post #5

> understanding. We evaluate on a large suite of test programs. Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler, 3% code size reduction is really good. The challenge will be having codegen like this that someone is willing to support. And for that they'd want to be able to reason about why the compiler made this decision or that one. IIUC that's an outstanding problem for AI…

Also bugs from this approach are going to be funny - program compiled with compiler version X will work as expected and same program compiled with version X+1 will start crashing because AI under some circumstances decided that dereference of a specific pointer was unnecessary, so it won't drop it into the assembly.

Good luck finding such a bug, because you will be looking on correct code, but computer will be executing invalid output.

Re: Large Language Models for Compiler Optimization

#14

Post by haltist »

Next step is to add verification for optimized code from the LLM with an SMT solver (like Z3) to remove "hallucinations". If the input and output code can be verified to be equivalent then this would be a great addition to an optimization pipeline. Once that's done the same can be applied to intermediate representations of GPU kernels in a recursive loop of AI optimizing AI code for faster execution times.

Re: Large Language Models for Compiler Optimization

#15

Post by a_wild_dandan »

TheLoafOfBread wrote: post #13
wyldfire wrote: post #5

> understanding. We evaluate on a large suite of test programs. Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler, 3% code size reduction is really good. The challenge will be having codegen like this that someone is willing to support. And for that they'd want to be able to reason about why the compiler made this decision or that one. IIUC that's an outstanding problem for AI…

Also bugs from this approach are going to be funny - program compiled with compiler version X will work as expected and same program compiled with version X+1 will start crashing because AI under some circumstances decided that dereference of a specific pointer was unnecessary, so it won't drop it into the assembly. Good luck finding such a bug, because you will be looking on correct code, but computer will be execut…

Damn. These things imitate humans too well. Guess we’ll need a giant test suite to feel stable. Sounds like work. Let’s train a GAN for that and put our feet up. Turtles all the way down, my dudes.

Re: Large Language Models for Compiler Optimization

#16

Post by [deleted] »

haltist wrote: post #14

Next step is to add verification for optimized code from the LLM with an SMT solver (like Z3) to remove "hallucinations". If the input and output code can be verified to be equivalent then this would be a great addition to an optimization pipeline. Once that's done the same can be applied to intermediate representations of GPU kernels in a recursive loop of AI optimizing AI code for faster execution times.

[deleted]

Re: Large Language Models for Compiler Optimization

#17

Post by sagarm »

jebarker wrote: post #11
kukkamario wrote: post #7

Earlier quoted context omitted.

They are not using LLM to directly produce the result code, but as tool that lists which optimisations should be done and in which order, which is fairly complex problem to solve. But if optimisation passes are implemented correctly (which is anyway required for a functioning optimising compiler), it cannot produce incorrect code, maybe only suboptimal compared to default heuristics used.

If there's a list of known optimizations that preserve correctness then it becomes an optimization problem based on output length (as a proxy for cycle count). So is the idea that an LLM is more efficient than a search or direct optimization?

There tend to be a lot of heuristics involved when deciding which optimizations to apply and in which order, so there's plenty of room to apply some machine learning.

Whether LLMs are the right approach is a separate question.

In SQL optimization, the problem is a bit trickier (IMO) because compilation is in the query path. One successful approach I know of is Bao: https://arxiv.org/abs/2004.03814

Re: Large Language Models for Compiler Optimization

#18

Post by [deleted] »

haltist wrote: post #14

Next step is to add verification for optimized code from the LLM with an SMT solver (like Z3) to remove "hallucinations". If the input and output code can be verified to be equivalent then this would be a great addition to an optimization pipeline. Once that's done the same can be applied to intermediate representations of GPU kernels in a recursive loop of AI optimizing AI code for faster execution times.

[deleted]

Re: Large Language Models for Compiler Optimization

#19

Post by boomanaiden154 »

CalChris wrote: post #4

There's a half day tutorial at the LLVM Developers Meeting on this, ML-Guided Compiler Optimization in LLVM . However, the authors of this paper aren't giving that tutorial.

It's a workshop intended to facilitate discussion in the space. The lead author of this paper (Chris Cummins) will probably be there.

https://discourse.llvm.org/t/pre-llvm-dev23-ml-guided-compil...

Re: Large Language Models for Compiler Optimization

#20

Post by jebarker »

reic wrote: post #10
jebarker wrote: post #3

Earlier quoted context omitted.

It seems like a poor fit to me precisely because correctness is boolean, difficult to measure and getting it wrong is very bad. I do think there's a place for AI here but it's probably not LLMs in their current form.

Quantification can be done by measuring in at least two dimensions: (1) the size of the synthesised code, and (2) how precisely the generated code matches the input (which means roughly: on what fraction of input do the two programs give different output). We have set up a challenge that seeks to entice the community to look into this problem domain more. And we've simplified the assumptions, so as to make it more tr…

How well does (2) really measure accuracy? It seems like a single output that doesn't match the input code could indicate a fundamental floor in the optimized code, so it's essentially 100% wrong even though it gets the correct answer almost all the time.

Good luck on the challenge though, this seems like an interesting and valuable area of research.

Post reply on HN