Couple of weeks ago I published a new programming language called Plang (as in pseudo language) that uses LLM to translate user intent into executable code, basically LLM as a compiler. It saves you incredible amount of work, cutting code writing down by 90%+. The built code is deterministic(it will never change after build) and as a programmer you can validate the code that will be executed. It compiles to C#, so it…
Language models as compilers: Simulating pseudocode execution
41–50 of 57 posts
Re: Language models as compilers: Simulating pseudocode execution
#42Earlier quoted context omitted.
Even regular compilers need quite a bit of nudging to give deterministic results.
Correct me if I'm wrong here, but I am under the impression they're only non-deterministic in the practical sense (i.e, it produces this output on my machine, I can't know what minute differences there are on your machine), but that's not non-deterministic in the truest sense. If you have completely identical inputs you will get the exact same output, ergo, deterministic.
Re: Language models as compilers: Simulating pseudocode execution
#43If you train a LLM to compile, you probably also want to set the randomness to zero, if that is the case you’ve just “brute forced” an actual compiler
you don't want a creative compiler?
Re: Language models as compilers: Simulating pseudocode execution
#44Couple of weeks ago I published a new programming language called Plang (as in pseudo language) that uses LLM to translate user intent into executable code, basically LLM as a compiler. It saves you incredible amount of work, cutting code writing down by 90%+. The built code is deterministic(it will never change after build) and as a programmer you can validate the code that will be executed. It compiles to C#, so it…
In my experience it’s not exactly trivial to validate code you didn’t write yourself, because you have to think through it in similar depth to when you write it yourself. While on the one hand you save the time of coming up with a solution, the task of merely verifying an existing solution is also more tedious, because it isn’t intermixed with the problem-solving activity that you perform when writing the code yourse…
That is why I design the language the way it is. You must define each step you want to happen in your application. Lets take for example user registration, it looks like this
--- plang code ---
CreateUser
- Make sure %password% and %email% is not empty
- Hash %password%, write to %hashedPassword%
- Insert into users, %hashedPassword%, %email%
- Post, create user in MailChimp, Bearer:%Settings.MailChimpApi%, %email%
- Create bearer token from %email%, write to %bearer%
- Write %bearer% to web response
--- plang code ---
That is an executable code in plang. It's easy to read through and understand. You need to have domain knowledge, such as what is hashing and bearer token. You are still programming, just at higher level.
Validating what will execute, you need to learn, just like with any language, but it is relatively simple and you start to trust the result with time(at least I have)
Compared to the 130 lines or so of code in C# for the same logic, https://gist.github.com/ingig/491ac9b13d65f40cc24ee5aed0408b... That´s about 95% reduction of code, and I see this repeatedly.
Re: Language models as compilers: Simulating pseudocode execution
#45Non deterministic compilers, yay! Where do I sign up? In more seriousness, miscompilations or in general unexpected behavior caused by layers below you are expensive to find and fix. I think LLMs have a long way to go before such use cases seem appealing to me.
It's better to have a non-deterministic compiler for a task that would be really hard to write an algorithm for otherwise.
Re: Language models as compilers: Simulating pseudocode execution
#46Non deterministic compilers, yay! Where do I sign up? In more seriousness, miscompilations or in general unexpected behavior caused by layers below you are expensive to find and fix. I think LLMs have a long way to go before such use cases seem appealing to me.
Re: Language models as compilers: Simulating pseudocode execution
#47Earlier quoted context omitted.
Even regular compilers need quite a bit of nudging to give deterministic results.
Correct me if I'm wrong here, but I am under the impression they're only non-deterministic in the practical sense (i.e, it produces this output on my machine, I can't know what minute differences there are on your machine), but that's not non-deterministic in the truest sense. If you have completely identical inputs you will get the exact same output, ergo, deterministic.
Re: Language models as compilers: Simulating pseudocode execution
#48The authors propose using an LLM to reframe the task as high level psuedocode, and then reason on that code on the specific details of the task
No compilers were used or compiled - no real code was generated or executed. Its just the idea that a programming language syntax has good structure to process details, and a way to interpret some of the results. Many of the other comments here seem like they didn't read the paper at all and are reacting to the headline
Re: Language models as compilers: Simulating pseudocode execution
#49Earlier quoted context omitted.
It's better to have a non-deterministic compiler for a task that would be really hard to write an algorithm for otherwise.
But are LLMs really better at algorithm writing than you? I’ve found that they work best when I’ve already pseudocoded the algorithm.
That's why I find non-determinism as acceptable when otherwise it would be a pain to do something similar.