11–20 of 57 posts
I do wonder how long hacks like this will be necessary; as it stands, many of these prompting techniques are essentially artificially expanding the input to enhance reasoning ability (increasing tokens, thus increasing chance of success).
Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
I envision a thread of research in the exact opposite direction. Take a logic program/theorem/database query/... and use a LLM to guide a search for a solution/proof/query plan/...
Earlier quoted context omitted.
I envision a thread of research in the exact opposite direction. Take a logic program/theorem/database query/... and use a LLM to guide a search for a solution/proof/query plan/...
LLM the Ultimate Heuristic?
Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
https://m.youtube.com/watch?v=PdE-waSx-d8
He’s very much on this kind of beat. In general I have a feeling that there are orders of magnitude to gain by successfully applying computer science to “language algorithms”.
Feels like we are exploring very narrow paths of computations that can be performed with language. Like we have an x86 cpu and we are building pocket calculators. So much untapped
Re prolog I had a similar intuition at some point and tried to make a stack based programming language that uses a language model as a kind of control/logic unit
https://github.com/LachlanGray/silas
I was missing a bunch of cs background at the time so I didn’t get very far, but I feel like there’s a lot to be done for this kind of thing
English is terribly imprecise, so it makes sense to use pseudo instructions to improve the bounds/outcome of a language model’s execution. I do wonder how long hacks like this will be necessary; as it stands, many of these prompting techniques are essentially artificially expanding the input to enhance reasoning ability (increasing tokens, thus increasing chance of success).
Another (and perhaps the ultimate) possibility is to have some way —- perhaps through simulations —- to directly expose the model to the problem, rather than having a human/natural language intermediary.
Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
For example, if you tell it to reply in JSON (and it obeys), you've just constrained its search space in a particular way. There is space for very interesting informal programming that can be done from this perspective, setting up constraints and then allowing inference to solve within them. I've been using this heavily.
When I was first getting deep into LLM stuff a few months ago and contemplating latent space my main characterization was that much of its high level behavior can be usefully grappled with by viewing it as a kind of 'learned geometric prolog'.
I did a bunch of illustrations and talked about some of these ideas here if anyone's curious: https://x.com/Westoncb/status/1757910205478703277 (I think I mostly dropped the prolog terminology in that presentation because not everyone knows about it)
If you train a LLM to compile, you probably also want to set the randomness to zero, if that is the case you’ve just “brute forced” an actual compiler
Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.
Quote:
One Cyc application aims to help students doing math at a 6th grade level, helping them much more deeply understand that subject matter... Unlike almost all other educational software, where the computer plays the role of the teacher, this application of Cyc, called MathCraft, has Cyc play the role of a fellow student who is always slightly more confused than you, the user, are about the subject.
Cyc is the huge reasoning engine. You can call it Prolog, if you want. I won't.
In more seriousness, miscompilations or in general unexpected behavior caused by layers below you are expensive to find and fix. I think LLMs have a long way to go before such use cases seem appealing to me.
Any sufficiently advanced LLM is indistinguishable from Prolog. I half-jest but I envision the direction of LLM research to head towards a parser-oriented setup where LLMs merely extract the entities and relations and the actual logic is done by a logical engine such as Prolog.