Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Large Language Models for Compiler Optimization

arxiv.org

Post reply on HN
Print view View original

91–100 of 119 posts

Re: Large Language Models for Compiler Optimization

#91

Post by ncruces »

wyldfire wrote: post #36
boomanaiden154 wrote: post #23

Earlier quoted context omitted.

The focus of this work is finding the optimal ordering of optimization passes to perform, not doing neural compilation. This guarantees correct code, assuming the underlying transformation passes are correct. Most work in ML for compilers focuses on replacing heuristics and phase ordering precisely because they don't impact correctness. There is some work being done on neural compilation [1], but I'm not sure that's…

> phase ordering precisely because they don't impact correctness. lol let's say they're less likely to impact correctness than an arbitrary new optimization.

If you find a bug, it's in the optimization passes (that don't reorder and should, if you decide to do this).

At this point this is like (usefully) fuzzing your optimizer, which long term is going to be great for correctness.

Re: Large Language Models for Compiler Optimization

#92

Post by aratauto »

smusamashah wrote: post #90

Do LLMs need to see some information just once to answer about it? My understanding was that they need to see many examples of the same thing to be able to answer it correctly (generate text around it correctly). Shouldn't this mean that it is safe to ask GPTs about something proprietary if it was just once because rare examples should just disappear in weights of everything else. And this also means that even GPT4 w…

Some recent discoveries indicate that LLMs might remember information after seeing it only once. See e.g.: https://www.fast.ai/posts/2023-09-04-learning-jumps/

Re: Large Language Models for Compiler Optimization

#93

Post by eru »

PhilipRoman wrote: post #83

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order. Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages…

> You don't ask the model what instructions to emit.

You could still do that, you'd just also need to ask the model for a proof. (But I guess that's much harder than heuristically picking which passes to apply.)

Re: Large Language Models for Compiler Optimization

#94

Post by SiempreViernes »

eru wrote: post #93
PhilipRoman wrote: post #83

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order. Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages…

> You don't ask the model what instructions to emit. You could still do that, you'd just also need to ask the model for a proof. (But I guess that's much harder than heuristically picking which passes to apply.)

What do you do when you check the proof and discover it is false? Just ask again for a proof or discard the emitted instructions?

Re: Large Language Models for Compiler Optimization

#95

Post by Lynix48 »

I was thinking about writing my bachelor's thesis about LLMs and compilers. My idea is to maybe attempt to write a C (to assembler)-Compiler and see how well it does.

Attempting to convert assembler to C using an LLM would be interesting too, but the results would probably be poor since there is just so much information that gets lost when compiling, so the C code would be pretty statistical. I guess I could improve it by somehow adding "C code is or is not compilable because of line n" and whether the result is correct to the lost function.

What do you guys think about this?

Re: Large Language Models for Compiler Optimization

#96

Post by JonChesterfield »

PhilipRoman wrote: post #83

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order. Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages…

> prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them)

Hundreds of passes, sure. Guaranteed that they preserve correctness is a bit more dubious, that's pretty hard to establish for most transforms. Passes that make no assumptions about prior passes are tricky too since compilers tend to work in terms of a lowering pipeline.

If the compiler has N correct passes that can be combined in arbitrary order without compromising compiler termination, exponentially increasing code size or generally making the output much worse, then you've already built a really good compiler. The subtask of then shuffling the order of passes to see if you missed anything is trivial, using machine learning to control your sort & test loop doesn't seem very compelling here.

My hunch is that the low hanging fruit in compiler dev using LLM is driving a fuzz tester with one. Other things seem worthwhile but difficult.

Re: Large Language Models for Compiler Optimization

#97

Post by JonChesterfield »

From the abstract:

- 3.0% improvement in reducing instruction counts over the compiler

- generating compilable code 91% of the time

- perfectly emulating the output of the compiler 70% of the time.

I read through the paper to see what perfectly emulating the output means. In this case, I think it's that it's also possible to get the same code out of the compiler using a different pass order. I was hoping for still passes original test suite or similar.

The authors are aware that the generated code has different semantics to the input in some cases but don't seem to consider that particularly important. The section is "Evaluation of Generated Code".

So - using machine learning, it is possible to delete a small percentage of the instructions emitted by a compiler while breaking the semantics. A tool which miscompiles programs while making them slightly smaller doesn't seem to be progress in compiler optimization.

Does anyone see some value add here that I'm missing?

Re: Large Language Models for Compiler Optimization

#98

Post by JonChesterfield »

wyldfire wrote: post #5

> understanding. We evaluate on a large suite of test programs. Our approach achieves a 3.0% improvement in reducing instruction counts over the compiler, 3% code size reduction is really good. The challenge will be having codegen like this that someone is willing to support. And for that they'd want to be able to reason about why the compiler made this decision or that one. IIUC that's an outstanding problem for AI…

3% code size reduction while changing semantics is borderline worthless*.

3% code size reduction without changing semantics would be more interesting but might still be a bad thing for performance.

*fast-math etc is a thing, where similar-enough output is fine

Re: Large Language Models for Compiler Optimization

#99

Post by titzer »

JonChesterfield wrote: post #96
PhilipRoman wrote: post #83

I see a lot of misconceptions about using ML for compilers. You don't ask the model what instructions to emit. Instead, you prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them). Then you ask the model - what passes should I apply and in what order. Writing code to unroll a loop is trivial. The limitations of compilers are that almost all currently existing languages…

> prepare a set of passes which are guaranteed to preserve correctness (we already have hundreds of them) Hundreds of passes, sure. Guaranteed that they preserve correctness is a bit more dubious, that's pretty hard to establish for most transforms. Passes that make no assumptions about prior passes are tricky too since compilers tend to work in terms of a lowering pipeline. If the compiler has N correct passes that…

> The subtask of then shuffling the order of passes to see if you missed anything is trivial, using machine learning to control your sort & test loop doesn't seem very compelling here.

This is called the "phase ordering problem", and it's neither trivial nor solved.

Re: Large Language Models for Compiler Optimization

#100

Post by eru »

SiempreViernes wrote: post #94
eru wrote: post #93

Earlier quoted context omitted.

> You don't ask the model what instructions to emit. You could still do that, you'd just also need to ask the model for a proof. (But I guess that's much harder than heuristically picking which passes to apply.)

What do you do when you check the proof and discover it is false? Just ask again for a proof or discard the emitted instructions?

Depends on your use case, I guess?
Post reply on HN