21–27 of 27 posts
Note that this refers to ML as in machine learning and not the ML language family, as I hoped.
There are so many flavors of machine learning that we should probably just refer to whole field as XML.
I know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly changing semantics at a layer that is unobservable.
I know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly changing semantics at a layer that is unobservable.
This sort of work isn't going "i dunno, emit some new instructions based on a model and hope it is correct." ML techniques fall into one of two categories: 1. Decisions that have no semantic change in the program but affect performance. This is things like code layout or register allocation that will be more cache friendly. This is what I expect to show up in industrial optimizing compilers more and more over the nex…
Yeah the problem is that there are not a whole lot of those in PL.
I know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly changing semantics at a layer that is unobservable.
WE "just" need to get the models outputting machine checkable proofs that semantics are preserved. I wonder what like this are going on in the math scene.
That's quite a tall order. Godel and all that.
My research (ML for binary function recognition) uses obfuscation and diverse compilation for data augmentation. Intuitively, obfuscation is a form of anti-optimization. My hope is that it's differentiable, so perhaps given enough knowledge of obfuscation, an ML model can make corresponding optimizations to "undo" them, then for more performance, apply those same un-obfuscations to code that hasn't already been obfus…
I think this is a great line of research as it may solve the reasoning portion of AGI. The difficulty is that you would want to prove equivalence. LLMs can output stuff but for programs to execute you need to output correct stuff.
Most of the work I have read in this field is focused on finding potential malware or stolen copyrighted code, so false positives are no big deal. But you don't want anything less than 100% valid code coming out of your compiler.
Earlier quoted context omitted.
This sort of work isn't going "i dunno, emit some new instructions based on a model and hope it is correct." ML techniques fall into one of two categories: 1. Decisions that have no semantic change in the program but affect performance. This is things like code layout or register allocation that will be more cache friendly. This is what I expect to show up in industrial optimizing compilers more and more over the nex…
> Decisions that have no semantic change Yeah the problem is that there are not a whole lot of those in PL.
I know it is irrational - as compilers already perform optimizations that are non intuitive - but my lizard brain really doesn’t like the idea of ML possibly changing semantics at a layer that is unobservable.
This sort of work isn't going "i dunno, emit some new instructions based on a model and hope it is correct." ML techniques fall into one of two categories: 1. Decisions that have no semantic change in the program but affect performance. This is things like code layout or register allocation that will be more cache friendly. This is what I expect to show up in industrial optimizing compilers more and more over the nex…