Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I accidentally turned LLM memory into program analysis

pwning.systems

Post reply on HN
Print view View original

31–40 of 97 posts

Re: I accidentally turned LLM memory into program analysis

#31

Post by alansaber »

keeda wrote: post #5

Very cool. I recall an HN submission (which I can't find offhand unfortunately) that did something similar -- it used an LLM to decompose articles into a set of statements which were used to construct an entity-relationship graph of facts and events. It then queried that using conventional graph query methods, much like DataLog / Lemmalog is doing here. I remember it was particularly effective at answering timeline-b…

This is great for evidence grounding, but doesn't produce a large memory/reasoning improvement (in most cases)

Re: I accidentally turned LLM memory into program analysis

#33

Post by [deleted] »

sim04ful wrote: post #14

I reached a similar conclusion: LLMs should only really sit at the terminals of request fulfilment. 1. User request understanding: natural language -> a more rigorous representation, in my case Datalog. 2. Result interpretation: facts and derived facts -> natural language. Between those terminals, the work should be mechanical reasoning over some ontology or formal knowledge structure. That connects to another princi…

[deleted]

Re: I accidentally turned LLM memory into program analysis

#34

Post by egberts1 »

Limitation of LLM for and toward reverse engineering; it's the LLM innate error of forgetting states thru agentic recursion by overflow of context or prior premises being optimized away due to not using ternary-state (uninit/written/read) memory state.

Once again, on LLM being: a digital librarian, at its finest; logic a logic analyst, not so much.

Re: I accidentally turned LLM memory into program analysis

#36

Post by manmal »

alansaber wrote: post #31
keeda wrote: post #5

Very cool. I recall an HN submission (which I can't find offhand unfortunately) that did something similar -- it used an LLM to decompose articles into a set of statements which were used to construct an entity-relationship graph of facts and events. It then queried that using conventional graph query methods, much like DataLog / Lemmalog is doing here. I remember it was particularly effective at answering timeline-b…

This is great for evidence grounding, but doesn't produce a large memory/reasoning improvement (in most cases)

My problem with memory is that it goes stale, and updates to facts are often not changing all locations of that fact. Such a system should make it easier to maintain a single source of truth, and versioning, no?

Re: I accidentally turned LLM memory into program analysis

#37

Post by alansaber »

manmal wrote: post #36
alansaber wrote: post #31

Earlier quoted context omitted.

This is great for evidence grounding, but doesn't produce a large memory/reasoning improvement (in most cases)

My problem with memory is that it goes stale, and updates to facts are often not changing all locations of that fact. Such a system should make it easier to maintain a single source of truth, and versioning, no?

Yes you need a VCS and to recalculate relationships at certain thresholds. It adds a lot of complexity and computarional overhead, which is why there's no uniform widespread graph use

Re: I accidentally turned LLM memory into program analysis

#39

Post by mentalgear »

Animats wrote: post #26

So he's using an LLM to generate data stored in an "is_a" representation. That's so classic AI. Soon, he'll discover that he needs quantifiers. Then that "for all" is too strong sometimes, and he needs "for most". That way lies Cyc. It's not a bad idea. But it does have a history.

In general, what all the big LLM providers are doing is moving towards classical & neural (neuro-symbolic) AI - even though they dont publicly admit it because that would counter their claims for years of "scale is all you need" (which has vanished with diminishing returns, see $MS / altman's GPT-5 bet).
Post reply on HN