Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use Prolog to improve LLM's reasoning

shchegrikovich.substack.com

Post reply on HN
Print view View original

11–20 of 158 posts

Re: Use Prolog to improve LLM's reasoning

#11

Post by lukasb »

a1j9o94 wrote: post #7

I tried an experiment with this using a Prolog interpreter with GPT-4 to try to answer complex logic questions. I found that it was really difficult because the model didn't seem to know Prolog well enough to write a description of any complexity. It seems like you used an interpreter in the loop which is likely to help. I'd also be interested to see how o1 would do in a task like this or if it even makes sense to us…

I bet one person could probably build a pretty good synthetic NL->Prolog dataset. ROI for paying that person would be high if you were building a foundation model (ie benefits beyond being able to output Prolog.)

Re: Use Prolog to improve LLM's reasoning

#13

Post by hendler »

a1j9o94 wrote: post #7

I tried an experiment with this using a Prolog interpreter with GPT-4 to try to answer complex logic questions. I found that it was really difficult because the model didn't seem to know Prolog well enough to write a description of any complexity. It seems like you used an interpreter in the loop which is likely to help. I'd also be interested to see how o1 would do in a task like this or if it even makes sense to us…

I also wrote wrote an LLM to Prolog interpreter for a hackathon called "Logical". With a few hours effort I'm sure it could be improved.

https://github.com/Hendler/logical

I think while LLMs may approach completeness here, it's good to have an interpretable system to audit/verify and reproduce results.

Re: Use Prolog to improve LLM's reasoning

#14

Post by z5h »

i've come to appreciate, over the past 2 years of heavy Prolog use, that all coding should be (eventually) be done in Prolog.

It's one of few languages that is simultaneously a standalone logical formalism, and a standalone representation of computation. (With caveats and exceptions, I know). So a Prolog program can stand in as a document of all facts, rules and relations that a person/organization understands/declares to be true. Even if AI writes code for us, we should expect to have it presented and manipulated as a logical formalism.

Now if someone cares to argue that some other language/compiler is better at generating more performant code on certain architectures, then that person can declare their arguments in a logical formalism (Prolog) and we can use Prolog to translate between language representations, compile, optimize, etc.

Re: Use Prolog to improve LLM's reasoning

#17

Post by anthk »

nxobject wrote: post #8
tokinonagare wrote: post #4

Earlier quoted context omitted.

Missing some LISP but yeah it's funny how old things are new again (same story with wasm, RISC archs, etc.)

Lots of GOFAI being implemented again – decision trees, goal searching and planning, agent-based strategies... just not symbolic representations, and that might be the key. I figure you might get an interesting contribution out of skimming old AI laboratory publications and seeing whether you could find a way of implementing it through a single LLM, multiple LLM agents, methods of training, etc.

https://en.m.wikipedia.org/wiki/Constraint_satisfaction_prob...

Re: Use Prolog to improve LLM's reasoning

#20

Post by dmead »

z5h wrote: post #14

i've come to appreciate, over the past 2 years of heavy Prolog use, that all coding should be (eventually) be done in Prolog. It's one of few languages that is simultaneously a standalone logical formalism, and a standalone representation of computation. (With caveats and exceptions, I know). So a Prolog program can stand in as a document of all facts, rules and relations that a person/organization understands/declar…

It's taken ages for anything from functional programming to penetrate general use. Do you think uptake of logic stuff will be any faster?
Post reply on HN