Language models as compilers: Simulating pseudocode execution
1–10 of 57 posts
Re: Language models as compilers: Simulating pseudocode execution
#2This seems quite promising. Using pseudo-code as an intermediary step isn't new but seems like this takes it a bit further. Will need to see some code and test it out.
Re: Language models as compilers: Simulating pseudocode execution
#3The phase 2 prompt is complete, but the phase 3 prompt's initial part ends in "When constructing the main function, ...", and no mention of random seeds, so I guess this paper is not reproducible at all.
Re: Language models as compilers: Simulating pseudocode execution
#4It's going to be really fascinating to see this applied instead of chain of thought and other kinds of reasoning approaches, because it's generic. It should in principle work on every kind of LLM.
Re: Language models as compilers: Simulating pseudocode execution
#5Any sufficiently advanced LLM is indistinguishable from Prolog.
I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
Re: Language models as compilers: Simulating pseudocode execution
#6Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
In truth, I thought it would be a great idea to train them on turning English and code into formal specs and logic language. Then, we could use all our tools in those areas work from there.
Combining LLM’s with rewriting logic, like Maude or K Framework, would be the most, powerful option. The rewriting tools plus LLM’s could probably rapidly develop static analyzers and code porting tools.
Re: Language models as compilers: Simulating pseudocode execution
#7Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
Hehe, this was the exact thinking going across my mind in considering underlying architecture for a LLM OS/Word processor that was self aware of the documents across a project for consistency.
Re: Language models as compilers: Simulating pseudocode execution
#8Seeing this makes me want to reactivate an old project[0]. Been thinking more and more that LLMs could give it superpowers.
Re: Language models as compilers: Simulating pseudocode execution
#9Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
I envision a thread of research in the exact opposite direction. Take a logic program/theorem/database query/... and use a LLM to guide a search for a solution/proof/query plan/...
Re: Language models as compilers: Simulating pseudocode execution
#10If you train a LLM to compile, you probably also want to set the randomness to zero, if that is the case you’ve just “brute forced” an actual compiler