i've come to appreciate, over the past 2 years of heavy Prolog use, that all coding should be (eventually) be done in Prolog. It's one of few languages that is simultaneously a standalone logical formalism, and a standalone representation of computation. (With caveats and exceptions, I know). So a Prolog program can stand in as a document of all facts, rules and relations that a person/organization understands/declar…
Use Prolog to improve LLM's reasoning
71–80 of 158 posts
Re: Use Prolog to improve LLM's reasoning
#72Earlier quoted context omitted.
You’re in good company — the most influential AI academic of all time, the cooky grandfather of AI who picked up right where (when!) Turing left off, the man hated by both camps yet somehow in charge of them, agrees with you. I’m talking about Marvin Minsky, of course. See: Logical vs. Analogical (Minsky, 1991) https://ojs.aaai.org/aimagazine/index.php/aimagazine/article... …the limitations of current machine intelli…
Just throwing this out there for someone, "Scruffier Kind of Machinery" is a good name for a book, company or band.
Re: Use Prolog to improve LLM's reasoning
#73I tried an experiment with this using a Prolog interpreter with GPT-4 to try to answer complex logic questions. I found that it was really difficult because the model didn't seem to know Prolog well enough to write a description of any complexity. It seems like you used an interpreter in the loop which is likely to help. I'd also be interested to see how o1 would do in a task like this or if it even makes sense to us…
I also wrote wrote an LLM to Prolog interpreter for a hackathon called "Logical". With a few hours effort I'm sure it could be improved. https://github.com/Hendler/logical I think while LLMs may approach completeness here, it's good to have an interpretable system to audit/verify and reproduce results.
Re: Use Prolog to improve LLM's reasoning
#74I made a pipeline using Z3 (another prover language) to get LLMs to solve very specific puzzle problems: https://youtu.be/UjSf0rA1blc (and a presentation: https://youtu.be/TUAmfi8Ws1g ) Some thoughts: 1. Getting an LLM to model a problem accurately is a significant prompting exercise. Bridging casual logical statements and formal logic is difficult. E.g., "or" statements in English usually mean "xor" in logic. 2. Dom…
Re: Use Prolog to improve LLM's reasoning
#75Would Creating a prolog dataset would be beneficial, so that future LLM's can be trained on it and then they would be able to output prolog code.
Re: Use Prolog to improve LLM's reasoning
#76Re: Use Prolog to improve LLM's reasoning
#77An application I am developing for a customer needed to read constraints around clinical trials and essentially build a query from them. Constraints involve prior treatments, biomarkers, type of disease (cancers) etc. Using just an LLM did not produce reliable queries, despite trying many many prompts, so being an old Prolog hacker I wondered if using it might impose more 'logic' on the LLM. So we precede the textual…
Re: Use Prolog to improve LLM's reasoning
#78Does anyone know why US attorneys and law firms are not using Prolog-based apps to automate the low-hanging fruit of issue-spotting?
Re: Use Prolog to improve LLM's reasoning
#79An application I am developing for a customer needed to read constraints around clinical trials and essentially build a query from them. Constraints involve prior treatments, biomarkers, type of disease (cancers) etc. Using just an LLM did not produce reliable queries, despite trying many many prompts, so being an old Prolog hacker I wondered if using it might impose more 'logic' on the LLM. So we precede the textual…