Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use Prolog to improve LLM's reasoning

shchegrikovich.substack.com

Post reply on HN
Print view View original

151–158 of 158 posts

Re: Use Prolog to improve LLM's reasoning

#151

Post by YeGoblynQueenne »

upghost wrote: post #150
YeGoblynQueenne wrote: post #143

Earlier quoted context omitted.

It's a PhD thesis... It was put to me that nobody will ever read it and nobody cares what's in it, but it's my thesis and it matters to me :) Edit: you can see some of the results here btw: https://github.com/stassa/louise

Amazing!! There was a chapter on this in my Prolog AI [1] book but I quickly realized it was a superset of the difficulty of high level Prolog programming, high level symbolic AI, and complex composition, and I knew it would have to wait until I graduated from white belt Prolog, but I am incredibly excited to see a real project using this technique!! I remember when I went for my masters degree in AI/ML, the industry…

Oh, I didn't remember that Bratko had a chapter on ILP. I've met him in a couple of ILP conferences so I knew he's published work in ILP though [1]. The techniques described in the book are quite a bit older and, to be honest, they were rather limited, in particular with respect to learning recursion. There's a New Wave of ILP nowadays however and a flourishing of new approaches that followed from MIL, which kinda threw open the gates.

There's a recent synopsis of the latest advances in ILP here:

https://arxiv.org/pdf/2102.10556

>> With all the hot news in Prolog these days I'd think you should submit this!

You mean to HN? I guess I could. I tend to think HN will not find it particularly interesting. Go ahead and submit it yourself though if you feel like it :)

_________________

[1] One of those times Bratko told me that I'm good with Prolog. I replied that I'm going to be saying he said that to everyone who will listen for the rest of my life XD

Re: Use Prolog to improve LLM's reasoning

#152

Post by larodi »

bbor wrote: post #48
larodi wrote: post #40

Earlier quoted context omitted.

Been shouting here and many places for quite a while that CoT and all similar stuff eventually leads to logic programming. So happy I’m not crazy.

You’re in good company — the most influential AI academic of all time, the cooky grandfather of AI who picked up right where (when!) Turing left off, the man hated by both camps yet somehow in charge of them, agrees with you. I’m talking about Marvin Minsky, of course. See: Logical vs. Analogical (Minsky, 1991) https://ojs.aaai.org/aimagazine/index.php/aimagazine/article... …the limitations of current machine intelli…

UAI seems reasonable as we need both discreet and non-discreet systems to start talking in a meaningful way.

Was wondering recently - given that a lot can be done with predicate logic, and given that DNA is a sort of grammar, is there anything more powerful than these formalisms in math in genera, that is actually put to work somewhere, anywhere?

Re: Use Prolog to improve LLM's reasoning

#153

Post by larodi »

YeGoblynQueenne wrote: post #107
bbor wrote: post #48

Earlier quoted context omitted.

You’re in good company — the most influential AI academic of all time, the cooky grandfather of AI who picked up right where (when!) Turing left off, the man hated by both camps yet somehow in charge of them, agrees with you. I’m talking about Marvin Minsky, of course. See: Logical vs. Analogical (Minsky, 1991) https://ojs.aaai.org/aimagazine/index.php/aimagazine/article... …the limitations of current machine intelli…

Minsky is right. The "rift" between symbolic and sub-symbolic, or learning and reasoning has only impeded progress. The problem is it's very hard to be an expert on both at once, and it's getting harder and harder as more and more work is done on both. Since you mentioned Hinton, he has worked hard to entrench the idea that symbolic AI failed. I was listening to a lecture he gave [1] and he went on and On and ON abou…

You’d be astonished how correct is this. I know several top PHD ppl well versed in ML who openly admin they know nothing about SQL which implies they also know very little PROLOG and very likely are ignorant about everything that is grammars and state automata.

Top surprise was when my high school classmate who went on to win two gold medals in IoM and has been doing quant mathematics for finance more than 16years openly admitted he knew nothing about grammars and was like ‘is this useful at all…’. I was amazed how is this even possible. But it is - he went the probabilistic and symbolic way, I went the discret and graph way.

On the other side I’m completely oblivious of what people use complex analysis for, even though I know a little DSP, some electronics, even some nano opto-electronics, and also can explain Fourier Transform to people. Even though I know what dérivâtes, nabla and vector field is, I can’t put them to work for me…

Science is never done in isolation, and the whole LLM thing seems from another planet to many people cause it was devised in a ML silo and also enterprise silo.

Re: Use Prolog to improve LLM's reasoning

#154

Post by YeGoblynQueenne »

larodi wrote: post #153
YeGoblynQueenne wrote: post #107

Earlier quoted context omitted.

Minsky is right. The "rift" between symbolic and sub-symbolic, or learning and reasoning has only impeded progress. The problem is it's very hard to be an expert on both at once, and it's getting harder and harder as more and more work is done on both. Since you mentioned Hinton, he has worked hard to entrench the idea that symbolic AI failed. I was listening to a lecture he gave [1] and he went on and On and ON abou…

You’d be astonished how correct is this. I know several top PHD ppl well versed in ML who openly admin they know nothing about SQL which implies they also know very little PROLOG and very likely are ignorant about everything that is grammars and state automata. Top surprise was when my high school classmate who went on to win two gold medals in IoM and has been doing quant mathematics for finance more than 16years op…

>> You’d be astonished how correct is this.

No I know it first hand :)

I'm also not a cross-disciplinary expert, to be clear. When I say it's hard, it's because I find it hard! My strength is in discrete maths and logic. I can deal with continuous maths because I need to keep abreast with the latest statistical machine learning developments but I don't think I would ever be able to contribute directly to say neural networks research, unless I turned it into a logic-based approach (as has been done in the past). To be perfectly honest, if deep learning didn't happen to be the dominant approach to machine learning, which forces me to pay attention to it, I doubt I would have followed my own advice and looked far beyond my narrow band of expertise.

But, that's why we're supposed to have collaborations, right? I can pair up with an expert on neural nets and we can make something new together that's more than what we can each do on our own. In the process we can learn from each other. That stuff works, I've seen that, too, in practice. I'm working with some roboticists now and I've learned a hell lot about their discipline and hopefully they're learning something about mine. I am convinced that in order to make progress in AI we need broad and wide collaborations, and not just between symbolists and connectionists, but also between computer scientists and biologists, cognitive scientists, whoever has any idea about what we're trying to achieve. After all, a computer scientist can only tell you something about the "artificial" in "artificial intelligence". We study computation, not intelligence. If we're going to create artificial intelligence we need to collaborate with someone who understands what that is.

The hard part is to kick people out of their comfort zone and to convince them that the other experts are also, well, experts, and that they have useful knowledge and skills that can improve your own results. And seen from the other side of the coin, from my point of view, it's very difficult for me, as an early career researcher, to convince anyone that I have useful knowledge and skills and something to contribute. It takes time and you have to make your name somehow otherwise nobody will want to work with you. But that's how academia works.

It's just not a great mechanism to ensure knowledge is shared and reused, unfortunately.

Re: Use Prolog to improve LLM's reasoning

#155

Post by upghost »

YeGoblynQueenne wrote: post #151
upghost wrote: post #150

Earlier quoted context omitted.

Amazing!! There was a chapter on this in my Prolog AI [1] book but I quickly realized it was a superset of the difficulty of high level Prolog programming, high level symbolic AI, and complex composition, and I knew it would have to wait until I graduated from white belt Prolog, but I am incredibly excited to see a real project using this technique!! I remember when I went for my masters degree in AI/ML, the industry…

Oh, I didn't remember that Bratko had a chapter on ILP. I've met him in a couple of ILP conferences so I knew he's published work in ILP though [1]. The techniques described in the book are quite a bit older and, to be honest, they were rather limited, in particular with respect to learning recursion. There's a New Wave of ILP nowadays however and a flourishing of new approaches that followed from MIL, which kinda th…

YAY!! Thank you! I feel like there is "so much Prolog" that is not really widely known about, it's not like it's burning up the blogosphere unfortunately. I had no idea the ILP methods in the Bratko book were already so dated. Are these Prolog specific conferences you go to or general ML ones...? Where do all the cool kids hang out??

Re: Use Prolog to improve LLM's reasoning

#156

Post by z5h »

dmichulke wrote: post #67
z5h wrote: post #14

i've come to appreciate, over the past 2 years of heavy Prolog use, that all coding should be (eventually) be done in Prolog. It's one of few languages that is simultaneously a standalone logical formalism, and a standalone representation of computation. (With caveats and exceptions, I know). So a Prolog program can stand in as a document of all facts, rules and relations that a person/organization understands/declar…

> It's one of few languages that is simultaneously a standalone logical formalism, and a standalone representation of computation. (With caveats and exceptions, I know). Would you be able to formulate all those "caveats and exceptions" in Prolog?

For example: the logical core of Prolog along with it's resolution model (for the logical part) are non deterministic (something can have none, one, many solutions) but only one solution is explored at a time. So it's a "meta logical" thing to express something like "the set of solutions for ...". Given that the core of Prolog is Turing complete, you can still get Prolog to compute anything, you might just not have a nice way of declaring it in pure Prolog.

Prolog has an interesting history of people discovering ways to express things that are simple, powerful and elegant. And yet despite the simplicity, these ways of expressing things were not immediately evident. DCGs are a prime example.

Re: Use Prolog to improve LLM's reasoning

#157

Post by hendler »

shchegrikovich wrote: post #73
hendler wrote: post #13

Earlier quoted context omitted.

I also wrote wrote an LLM to Prolog interpreter for a hackathon called "Logical". With a few hours effort I'm sure it could be improved. https://github.com/Hendler/logical I think while LLMs may approach completeness here, it's good to have an interpretable system to audit/verify and reproduce results.

This is really cool!

Thanks! Feel free to reach out.

Re: Use Prolog to improve LLM's reasoning

#158

Post by vaguetruth »

guys! I'm into logic and philosophy of language applied to psychology. stumbled on this thread from googling prolog! please, can you recommend a programming Prolog video intro for me? I'm interested in programming (my undergrad is cs) immediately and applying Prolog to write psycho-philosophical case studies.
Post reply on HN