Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

61–70 of 119 posts

Re: Large Language Models for Compiler Optimization

#61

Post by haltist »

boomanaiden154 wrote: post #21
haltist wrote: post #14

Next step is to add verification for optimized code from the LLM with an SMT solver (like Z3) to remove "hallucinations". If the input and output code can be verified to be equivalent then this would be a great addition to an optimization pipeline. Once that's done the same can be applied to intermediate representations of GPU kernels in a recursive loop of AI optimizing AI code for faster execution times.

There's already tooling available for using SMT to validate LLVM-IR transformations [1]. It's designed for zero false positives however, so some things might slip through the cracks. Additionally, this work focuses on phase ordering, which produces correct code regardless of what the LLM puts out, assuming there aren't any bugs in the passes being used (which could crop up as random orderings aren't as well tested as…

Interesting. AI code has no loops so the problem they mention in the beginning about unrolling loops to a certain depth is a non-issue.

Re: Large Language Models for Compiler Optimization

#62

Post by naveen99 »

Izkata wrote: post #55
naveen99 wrote: post #48

Earlier quoted context omitted.

The toy example for those curious: #include int main(void) { int i; int a[2000] ; for(i = 0; i This program initializes an array `a` of size 2000, populates it with integers from 0 to 1999, and then prints the value at index 5, which is 5. Just as with the Python program, we can optimize this program significantly. Given that we're only interested in the sixth element (index 5) of the array, we don't need to construc…

Code indentation is 2 spaces on here, you have to do it to all the lines you want formatted that way. That's why the first one got a bunch of lines combined, it treated them as one paragraph instead of code.

thanks. fixed.

Re: Large Language Models for Compiler Optimization

#63

Post by quickthrower2 »

CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

Sounds like TDD and langchain?

Where the T can be anything from Ruby unit tests to Coq proofs.

Re: Large Language Models for Compiler Optimization

#64

Post by alex_lav »

reidjs wrote: post #35
alex_lav wrote: post #30

Earlier quoted context omitted.

At my company we have some "loadbearing" code that was written by a mad scientist that no longer works at our company. We have a total ban on AI for source code analysis (so chatgpt, copilot etc are all banned). I've really wanted to throw some of the grosser parts of that codebase into gpt-4 just to see if it could bring some small amount of sanity.

Do you know why that policy is in place? Is it fear that the llm provider will steal company trade knowledge?

I work in a research facility, so the biggest fear is that our super top secret elite info will leak. The reality is it'd be used to refactor a lot of terrible code.

Re: Large Language Models for Compiler Optimization

#65

Post by wrsh07 »

CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

Are you compiling the code or interpreting?

Re: Large Language Models for Compiler Optimization

#66

Post by darksaints »

CJefferson wrote: post #57

While this isn't optimization, is there any LLM systems that create "provably correct" transformations yet? I have used ChatGPT on some fairly awful code, asking it to add comments, rename variables and functions, etc. I find the outputs useful, but a couple of times it's broken the code. I imagine for many (not all) languages, you could ask the LLM to produce suggested changes, then use a code-rewriting tool to appl…

I can't imagine there would be any success with an LLM that isn't already known about via Satisfiability Modulo Theory.

Re: Large Language Models for Compiler Optimization

#67

Post by cypress66 »

naveen99 wrote: post #48
naveen99 wrote: post #29

Chatgpt4 can do source to source optimization which is pretty cool. I got it to beat gcc at -03 on simple small toy problems. and it can do similar things with python also. But it threw its hands up when I gave it a longer piece of code to optimize.

The toy example for those curious: #include int main(void) { int i; int a[2000] ; for(i = 0; i This program initializes an array `a` of size 2000, populates it with integers from 0 to 1999, and then prints the value at index 5, which is 5. Just as with the Python program, we can optimize this program significantly. Given that we're only interested in the sixth element (index 5) of the array, we don't need to construc…

Printf is horribly slow compared to puts

Re: Large Language Models for Compiler Optimization

#68

Post by cypress66 »

alex_lav wrote: post #64
reidjs wrote: post #35

Earlier quoted context omitted.

Do you know why that policy is in place? Is it fear that the llm provider will steal company trade knowledge?

I work in a research facility, so the biggest fear is that our super top secret elite info will leak. The reality is it'd be used to refactor a lot of terrible code.

Use local models if you don't want to send your data to OpenAI

Re: Large Language Models for Compiler Optimization

#69

Post by Buttons840 »

jebarker wrote: post #11
kukkamario wrote: post #7

Earlier quoted context omitted.

They are not using LLM to directly produce the result code, but as tool that lists which optimisations should be done and in which order, which is fairly complex problem to solve. But if optimisation passes are implemented correctly (which is anyway required for a functioning optimising compiler), it cannot produce incorrect code, maybe only suboptimal compared to default heuristics used.

If there's a list of known optimizations that preserve correctness then it becomes an optimization problem based on output length (as a proxy for cycle count). So is the idea that an LLM is more efficient than a search or direct optimization?

This is scooching into AlphaGo (or whatever the generic implementation is called) territory: mixing predictive AI with traditional optimization algorithms.

Re: Large Language Models for Compiler Optimization

#70

Post by jmgao »

cypress66 wrote: post #67
naveen99 wrote: post #48

Earlier quoted context omitted.

The toy example for those curious: #include int main(void) { int i; int a[2000] ; for(i = 0; i This program initializes an array `a` of size 2000, populates it with integers from 0 to 1999, and then prints the value at index 5, which is 5. Just as with the Python program, we can optimize this program significantly. Given that we're only interested in the sixth element (index 5) of the array, we don't need to construc…

Printf is horribly slow compared to puts

puts isn't applicable here because you actually need to format the %d, but any compiler worth using (i.e. not MSVC: https://gcc.godbolt.org/z/zx74vY1za) will optimize a printf of a constant string ending in a newline into a puts.
Post reply on HN