Live data from Hacker News

SymbolicAI: A neuro-symbolic perspective on LLMs

github.com

51–60 of 66 posts

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#51

Earlier quoted context omitted.

That's gonna be a very, very, long answer. What's funny is that not much changed since 2022 (eoy) when the projected started; the models just got better, but we had a good chunk of the primitives since gpt-3. What's more recent is the DbC contribution which I think is unique. It literally solved anything agent related I've thrown at it -- especially because I can chain contracts together and the guardrails propagate…

One last comment here on contracts; an excerpt from the linked post I think it's extremely relevant for LLMs, maybe it triggers an interesting discussion here: "The scope of contracts extends beyond basic validation. One key observation is that a contract is considered fulfilled if both the LLM’s input and output are successfully validated against their specifications. This leads to a deep implication: if two differe…

Anyone interested in this from a history / semiotics / language-theory perspective should look into the triad concepts of:

Sign (Signum) - The thing which points Locus - The thing being pointed to Sense (Sensus) - The effect/sense in the interpreter

Also known by: Representation/Object/Interpretation, Symbol/Referent/Thought, Signal/Data/User, Symbol/State/Update. Same pattern has been independently identified many many times through history, always ending up with the triplet, renamed many many times.

What you're describing above is the "Locus" essential object being pointed to, fulfilled by different contracts/LLMs/systems but the same essential thing always being eluded to. There's an elegant stability to it from a systems design pov. It makes strong sense to build around those as the indexes/keys being pointed towards, and then various implementations (Signs) attempting to achieve them. I'm building a similar system atm.

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#52

Earlier quoted context omitted.

One last comment here on contracts; an excerpt from the linked post I think it's extremely relevant for LLMs, maybe it triggers an interesting discussion here: "The scope of contracts extends beyond basic validation. One key observation is that a contract is considered fulfilled if both the LLM’s input and output are successfully validated against their specifications. This leads to a deep implication: if two differe…

Anyone interested in this from a history / semiotics / language-theory perspective should look into the triad concepts of: Sign (Signum) - The thing which points Locus - The thing being pointed to Sense (Sensus) - The effect/sense in the interpreter Also known by: Representation/Object/Interpretation, Symbol/Referent/Thought, Signal/Data/User, Symbol/State/Update. Same pattern has been independently identified many m…

Thanks for bringing this up. I'm fairly familiar with Peirce's triadic semiotics and Montague's semantics, and they show up in some of my notes. I haven't turned those sketches into anything applied yet, but the design space feels *huge* and quite promising intuitively.

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#53
post #43

What bums me out a bit - "Symbolic AI" is already well defined: https://en.m.wikipedia.org/wiki/Symbolic_artificial_intellig...

We hear you. We might end up renaming it. In the paper we have a footnote about the name choice -- it's meant to credit the foundational work of Newell and Simon that inspired this project.

I'd appreciate it! It's cool and I wish you success. Just hope that when someone says "We're using Symbolic AI" a year from now, it won't be even more ambiguous than today :D

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#54

One question, OP, how does cost for this work? Do you pay the LLM inference cost (quite literally if using an external API) every time you run a line that involves natural language computation? E.g. what happens if you call a "symbolic" function in a loop.

This will need a cache of some sort

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#55

I spent some time toying around with LLM-guided "symbolic regression", basically having an LLM review documents in order to come up with primitives (aka operators) that could be fed into github.com/MilesCranmer/PySR I didn't get very far because I had difficulty piping it all together, but with something like this I might give it another go. Cool stuff.

Oh, definitely. I recommend you go for contracts. I've used something similar for a contract that iteratively "stitched together" a broken ontology graph. Here's some of the data models for inspiration -- you could have something similar for your ops, and write the contract to solve for one op, then apply the op, etc.

---

    class Merge(LLMDataModel):
        indexes: list[int] = Field(description="The indices of the clusters that are being merged.")
        relations: list[SubClassRelation] = Field(
            description="A list of superclass-subclass relations chosen from the existing two clusters in such a way that they merge."
        )
    
        @field_validator("indexes")
        @classmethod
        def is_binary(cls, v):
            if len(v) != 2:
                raise ValueError(
                    f"Binary op error: Invalid number of clusters: {len(v)}. The merge operation requires exactly two clusters."
                )
            return v
    
    
    class Bridge(LLMDataModel):
        indexes: list[int] = Field(description="The indices of the clusters that are being bridged.")
        relations: list[SubClassRelation] = Field(
            description="A list of new superclass-subclass relations used to bridge the two clusters from the ontology."
        )
    
        @field_validator("indexes")
        @classmethod
        def is_binary(cls, v):
            if len(v) != 2:
                raise ValueError(
                    f"Binary op error: Invalid number of clusters: {len(v)}. The merge operation requires exactly two clusters."
                )
            return v
    
    
    class Prune(LLMDataModel):
        indexes: list[int] = Field(description="The indices of the clusters that are being pruned.")
        classes: list[str] = Field(description="A list of classes that are being pruned from the ontology.")
    
        @field_validator("indexes")
        @classmethod
        def is_unary(cls, v):
            if len(v) > 1:
                raise ValueError(
                    f"Unary op error: Invalid number of clusters: {len(v)}. The prune operation requires exactly one cluster."
                )
            return v
    
    
    class Operation(LLMDataModel):
        type: Merge | Bridge | Prune = Field(description="The type of operation to perform.")
---

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#57
post #30
post #29

Earlier quoted context omitted.

Why is carrot the vegetablefication of apple?

Also if you run it twice, is it gonna be a carrot again?

Since these seem like short prompts, you can send as context data that was correct on past prompts

You can create a test suite for your code that will compile correct results according to another prompt or dictionary verification

  t.test(
     Symbol(['apple', 'banana', 'cherry', 'cat', 'dog']).map('convert all fruits to vegetables'),
     "list only has vegetable and cat,dog"
  )

Re: SymbolicAI: A neuro-symbolic perspective on LLMs

#58

Earlier quoted context omitted.

Anyone interested in this from a history / semiotics / language-theory perspective should look into the triad concepts of: Sign (Signum) - The thing which points Locus - The thing being pointed to Sense (Sensus) - The effect/sense in the interpreter Also known by: Representation/Object/Interpretation, Symbol/Referent/Thought, Signal/Data/User, Symbol/State/Update. Same pattern has been independently identified many m…

Thanks for bringing this up. I'm fairly familiar with Peirce's triadic semiotics and Montague's semantics, and they show up in some of my notes. I haven't turned those sketches into anything applied yet, but the design space feels *huge* and quite promising intuitively.

Agreed. This is a very interesting discussion! Thanks for bringing it to light.

Have you read Escher, Bach, Gödel: the Eternal Golden Braid?

Post reply on HN