Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

δ-mem: Efficient Online Memory for Large Language Models

arxiv.org

Post reply on HN
Print view View original

41–50 of 69 posts

Re: δ-mem: Efficient Online Memory for Large Language Models

#41

Post by DeathArrow »

stephantul wrote: post #5
DeathArrow wrote: post #2

I see lots of techniques proposed to give LLM the capacity to recall things, I even saw a lot of memory plugins for AI coding agents, I tried some myself. What I want to see is something that was tested and proved in practice to be genuinely useful, especially for coding agents.

How would you conceptualize recall in this case? Is searching through the current version of your code and possibly git history not enough?

>Is searching through the current version of your code and possibly git history not enough?

While you can document everything and use git history, I think that having short entries in a kind of memory to remember past decisions, how issues were solved would be much more token efficient than reading lots of documentation and looking at git history and past code.

Re: δ-mem: Efficient Online Memory for Large Language Models

#42

Post by cubefox »

MeteorMarc wrote: post #18
cubefox wrote: post #16

Papers being voted high on Hacker News are usually uncorrelated with their actual importance. It's basically a lottery. There are regularly more interesting papers going semi viral on Twitter.

On huggingface it was #3 paper of the day, which is neutral towards your hypothesis.

Considering that there is a paper with this many points perhaps once a week here (probably less), #3 of the day is pretty unremarkable.

Re: δ-mem: Efficient Online Memory for Large Language Models

#46

Post by in-silico »

usernametaken29 wrote: post #11

> δ-mem compresses past information into a fixed-size state matrix updated by delta-rule learning This doesn’t solve the capacity problem of memory. You can cram more into one context window, but then again you need to associate them with input queries. That’s very hard because slight variations in input create hugely different activations. So really, it doesn’t improve caching. This paper might do a thing or two app…

While there is a limit to the amount of information you can fit in a fixed-size state, the theoretical ceiling is pretty high.

A Hebbian associative matrix (one of the simplest and weakest memory constructions) can store about 0.7 bits of information per parameter. If you have a state with 300M parameters (the size of a Llama 3 8B KV cache at 10K context length), and a context with 2.1 bits of entropy per token (a reasonable estimate), then the state can encode 100M tokens worth of information.

Real models obviously aren't powerful enough to operate at the limit, but you can see why this is a promising research direction.

Re: δ-mem: Efficient Online Memory for Large Language Models

#48

Post by airstrike »

realitysballs wrote: post #39
airstrike wrote: post #33

Earlier quoted context omitted.

The submitter has a grace period of a few minutes to edit the title after submitting, so there's no need to change what HN does

True, but wouldn’t it be better long term if website automation didn’t create unintended new meanings to Titles? title’s matter

Only if you assume it doesn't ever work as intended.

Re: δ-mem: Efficient Online Memory for Large Language Models

#49

Post by airstrike »

maxignol wrote: post #47

Is there some kind of memory enabling, for instance, an agent to remember guidelines on a repo without having to feed at the beginning of each session 4 markdown files and spending the corresponding tokens each time ?

No, it's all just prompts.

You can try to summarize memories tersely and point the agent to longer markdown files, but who knows if it will read it at the right time and only then.

Re: δ-mem: Efficient Online Memory for Large Language Models

#50

Post by throw1234567891 »

realitysballs wrote: post #39
airstrike wrote: post #33

Earlier quoted context omitted.

The submitter has a grace period of a few minutes to edit the title after submitting, so there's no need to change what HN does

True, but wouldn’t it be better long term if website automation didn’t create unintended new meanings to Titles? title’s matter

indeed, titles matter
Post reply on HN