Use Prolog to improve LLM's reasoning
shchegrikovich.substack.com
Use Prolog to improve LLM's reasoning
1–10 of 158 posts
Re: Use Prolog to improve LLM's reasoning
#2So we are back to Japanese Fifth Generation plan from 1980's. :)
Re: Use Prolog to improve LLM's reasoning
#3So we are back to Japanese Fifth Generation plan from 1980's. :)
Watson did it too, a while back.
Re: Use Prolog to improve LLM's reasoning
#4So we are back to Japanese Fifth Generation plan from 1980's. :)
Missing some LISP but yeah it's funny how old things are new again (same story with wasm, RISC archs, etc.)
Re: Use Prolog to improve LLM's reasoning
#5Building on this idea people have grounded LLM generated reasoning logic with perceptual information from other networks : https://web.stanford.edu/~joycj/projects/left_neurips_2023
Re: Use Prolog to improve LLM's reasoning
#6So we are back to Japanese Fifth Generation plan from 1980's. :)
This time around we have all sorts of parallel processing capabilities in the form of GPUs. If I recall correctly, the Fifth Generation project envisioned highly parallel machines performing symbolic AI. From a hardware standpoint, those researchers were way ahead of their time.
Re: Use Prolog to improve LLM's reasoning
#7I tried an experiment with this using a Prolog interpreter with GPT-4 to try to answer complex logic questions. I found that it was really difficult because the model didn't seem to know Prolog well enough to write a description of any complexity.
It seems like you used an interpreter in the loop which is likely to help. I'd also be interested to see how o1 would do in a task like this or if it even makes sense to use something like prolog if the models can backtrack during the "thinking" phase
Re: Use Prolog to improve LLM's reasoning
#8So we are back to Japanese Fifth Generation plan from 1980's. :)
Missing some LISP but yeah it's funny how old things are new again (same story with wasm, RISC archs, etc.)
Lots of GOFAI being implemented again – decision trees, goal searching and planning, agent-based strategies... just not symbolic representations, and that might be the key. I figure you might get an interesting contribution out of skimming old AI laboratory publications and seeing whether you could find a way of implementing it through a single LLM, multiple LLM agents, methods of training, etc.
Re: Use Prolog to improve LLM's reasoning
#9Patiently waiting for z3-guided generation, but this is a welcome, if obvious, development. Results are a bit surprising and sound too optimistic, though.
Re: Use Prolog to improve LLM's reasoning
#10So we are back to Japanese Fifth Generation plan from 1980's. :)
This time around we have all sorts of parallel processing capabilities in the form of GPUs. If I recall correctly, the Fifth Generation project envisioned highly parallel machines performing symbolic AI. From a hardware standpoint, those researchers were way ahead of their time.
And they had a self-sustaining video game industry too... if only someone had had the wild thought of implementing perceptrons and tensor arithmetic on the same hardware!