Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Use Prolog to improve LLM's reasoning

shchegrikovich.substack.com

Post reply on HN
Print view View original

131–140 of 158 posts

Re: Use Prolog to improve LLM's reasoning

#131

Post by sgdfhijfgsdfgds »

YeGoblynQueenne wrote: post #92
Philpax wrote: post #42

Earlier quoted context omitted.

As deep as is required to actually make your argument!

You'll have to be more specific than that. For me what I point out is obvious: Prolog is not magick. Your program won't magickally reason if you write it in Prolog, much less reason correctly. If an LLM translates a Problem to the wrong Prolog program, Prolog won't magickally turn it into a correct program. And that's just rephrasing what I said in my comment above. There's really not much more to say. Here's just on…

> Again, Prolog is not magick. The article above and the papers it links to seem to take this attitude of "just add Prolog" and that will make LLMs suddenly magickally reason with fairy dust on top. Ain't gonna happen.

It frightens me that HN is so popular with people who will strain credulity in this regard. It's like a whole decade of people engaging in cosmic-ordering wishes about crypto has now led to those same people wishing for new things as if the wishes themselves are evidence of future outcomes.

Re: Use Prolog to improve LLM's reasoning

#132

Post by sgdfhijfgsdfgds »

shchegrikovich wrote: post #129
sgdfhijfgsdfgds wrote: post #126

Earlier quoted context omitted.

I know we're not supposed to comment on downvotes but I really question the logic of anyone who thinks that a thing that cannot reason can write a prolog program that is really going to be much more successful. Prolog is actually pretty difficult to do right, even if you are skilled. It actually requires reasoning. You don't just write out facts and have the system do the work. And many of the examples in the trainin…

I have another example - just a few people believed that you can apply 'a simple next token prediction algorithm' and achieve what we know as LLM. From my perspective, in the past few years, we've tried a lot of different approaches to improve LLM reasoning; some of them were good, others not so good. We need to keep trying and researching. 'Prolog + LLM' is not the answer to all questions, but it looks like a good s…

> 'Prolog + LLM' is not the answer to all questions, but it looks like a good step to move us forward.

Or it's a thing people can write papers about, and chase reproducibility on afterwards, as the shell game of claiming LLM reasoning continues.

Re: Use Prolog to improve LLM's reasoning

#133

Post by sgdfhijfgsdfgds »

timonoko wrote: post #59

Interest in Prolog always ends with the "!". It is ugly and like smack in the head, "you are thinking too much".

The course I did at uni, decades ago now, set us a Prolog assessment where we were not allowed to use the cut operator.

Code that backtracks is hard to reason about.

Re: Use Prolog to improve LLM's reasoning

#135

Post by johnnyjeans »

dragonwriter wrote: post #85
johnnyjeans wrote: post #33

Earlier quoted context omitted.

Prolog (and logic programming in general) is much older than you think. In fact, if we take modern functional programming to have been born with John Backus' Turing Award presentation[1], then it even predates it. Many advancements to functional programming were implemented on top of Prolog! Erlang's early versions were built on top of a Prolog-derived language who's name escapes me. It's the source of Erlang's unfam…

> Erlang's early versions were built on top of a Prolog-derived language who's name escapes me. AFAIK, Erlang was originally implemented in Prolog and the original VM was inspired by the Warren Abstract Machine targeted by some Prolog implementations. It was also inspired by PLEX, but PLEX wasn't a Prolog derivative.

Oh, apparently I only had half the story. I remembered reading that it was written in a committed-choice derivative of Prolog for concurrency reasons, but apparently that was just a failed experiment that happened early on.[1]

[1] - https://erlangforums.com/t/is-the-first-version-of-erlang-wr...

Re: Use Prolog to improve LLM's reasoning

#136

Post by samatman »

treetalker wrote: post #60

Does anyone know why US attorneys and law firms are not using Prolog-based apps to automate the low-hanging fruit of issue-spotting?

Law was particularly badly burned by the hype wave of "expert systems" in the 1970s and 80s, many of them coded in Prolog.

https://en.wikipedia.org/wiki/Expert_system

The Wiki link is nearly hagiographic in its studied avoidance of the topic of how the field crashed and burned, and the term "expert system" fell into disrepute, but these are things which happened.

Which isn't to say that the legal field can't benefit from software which uses Prolog, in fact, I have a strong hunch that the number of such products currently in use is not zero.

But if you wrote a new one, you would do well to make sure that no senior partners hear the word "Prolog" in the sales pitch.

Re: Use Prolog to improve LLM's reasoning

#137

Post by upghost »

rdtsc wrote: post #124
upghost wrote: post #102

Earlier quoted context omitted.

I sympathize. I nearly dropped Prolog for this reason until I learned about term_expansion/2 and goal_expansion/2. If what Prolog is doing you consider incorrect, _make it_ incorrect. DCGs can be used to convert any data structure to a sequence. Actually, they are capable of any graph to graph , so they could produce a sequence of commands. The oft-cited Markus Triska has some great work on this: https://youtu.be/vda…

I was watching some of the videos and in the intro Markus mentioned some of the most popular Prolog systems https://www.youtube.com/watch?v=8XUutFBbUrg SICStus, GNU Prolog, XSB, Ciao, Scryer, Traella, Tau but does not mention at all of SWI-Prolog. I remember using SWI-Prolog back in the day, did it somehow fall out favor, or is there some animosity between implementation and Markus is just not in the SWI-Prolog camp?

My 2 cents, it's hard for me to believe there is any animosity there.

You don't have to take my word for it, you can just see what he says and does. But I've seen nearly every one of his videos and blog posts, read through many previous comments on HN, there's not a single disparaging remark in any of them. In fact he even makes his philosophy clear in this post [1]:

  Things that I, at least, always do without even thinking about it. For instance, when I mention Scryer Prolog in Internet discussions, I always try to mention at least one other Prolog system too in a flattering way, out of respect and admiration for other systems and also to encourage more cooperation between systems.
What I gather is Markus strongly advocates for ISO compliant Prolog implementations and especially open source ones, because that's where his heart is right now. But one thing to remember is that Markus has also contributed 10s of thousands of lines of code to SWI (check out the author tag in many of the SWI libraries), has co-authored authored many papers with Jan Wielemaker, and there is plenty of professional respect there. This is like trying to understand the nuance reasons why Steven Hawking disagrees with Einstein (or whoever) on something. Probably in agreement about 99.999% of most things but strong disagreement on a 0.001% that probably doesn't matter much to you and I about whether or not black holes are Humperdink-Blazensort compliant or... waves hands stuff.

As you've seen from the comments though, SWI is an extremely successful, established system. Nearly every book and example you will read uses SWI. It has great libraries, great IDEs, FFI, embedding, support, documentation, all kinds of great stuff -- SWI is already REALLY successful, and for very good reasons!

So my guess is just trying to give a voice to the little guys who are up and coming and are in line philosophically with his beliefs about the language and open source. I personally would describe that as "support", I don't think I would use the word "animosity".

[1]: https://github.com/mthom/scryer-prolog/discussions/2605

Re: Use Prolog to improve LLM's reasoning

#138

Post by rdtsc »

upghost wrote: post #137
rdtsc wrote: post #124

Earlier quoted context omitted.

I was watching some of the videos and in the intro Markus mentioned some of the most popular Prolog systems https://www.youtube.com/watch?v=8XUutFBbUrg SICStus, GNU Prolog, XSB, Ciao, Scryer, Traella, Tau but does not mention at all of SWI-Prolog. I remember using SWI-Prolog back in the day, did it somehow fall out favor, or is there some animosity between implementation and Markus is just not in the SWI-Prolog camp?

My 2 cents, it's hard for me to believe there is any animosity there. You don't have to take my word for it, you can just see what he says and does. But I've seen nearly every one of his videos and blog posts, read through many previous comments on HN, there's not a single disparaging remark in any of them. In fact he even makes his philosophy clear in this post [1]: Things that I, at least, always do without even th…

> This is like trying to understand the nuance reasons why Steven Hawking disagrees with Einstein (or whoever) on something.

I like the analogy!

> So my guess is just trying to give a voice to the little guys who are up and coming and are in line philosophically with his beliefs about the language and open source.

That makes perfect sense. Thanks for answering! It's an interesting insight into the world of "Prologs" to a complete outsider. I had only used SWI-Prolog in the university and remember it having a variety of modules, web development libraries and other such things, so at least superficially to a novice, that was pretty impressive.

Re: Use Prolog to improve LLM's reasoning

#139

Post by UltraSane »

z5h wrote: post #14

i've come to appreciate, over the past 2 years of heavy Prolog use, that all coding should be (eventually) be done in Prolog. It's one of few languages that is simultaneously a standalone logical formalism, and a standalone representation of computation. (With caveats and exceptions, I know). So a Prolog program can stand in as a document of all facts, rules and relations that a person/organization understands/declar…

I'm curious what your thoughts are on this paper [0]Lessons from the evolution of the Batfish configuration analysis tool. Initially they used Datalog but have since migrated to binary decision diagrams for performance reasons, the ability to more accurately model semantics, and more deterministic execution.

[0]:https://dl.acm.org/doi/10.1145/3603269.3604866

Re: Use Prolog to improve LLM's reasoning

#140

Post by upghost »

YeGoblynQueenne wrote: post #127
upghost wrote: post #112

Earlier quoted context omitted.

can you link your thesis? That sounds awesome!!

I can't seem to find it on the web. I don't think it's published online yet.

that's ok (T_T)

I'll just imagine what it could've been like! I bet it was awesome!

Post reply on HN