Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

δ-mem: Efficient Online Memory for Large Language Models

arxiv.org

Post reply on HN
Print view View original

11–20 of 69 posts

Re: δ-mem: Efficient Online Memory for Large Language Models

#11

Post by usernametaken29 »

> δ-mem compresses past information into a fixed-size state matrix updated by delta-rule learning

This doesn’t solve the capacity problem of memory. You can cram more into one context window, but then again you need to associate them with input queries. That’s very hard because slight variations in input create hugely different activations. So really, it doesn’t improve caching. This paper might do a thing or two approximating the compression limit for context windows, but there’s a fundamental limit on how much information can go into it. What you really need is contextual search, as in, different events and objects with the same abstractions and semantic lead to same response, so you can cache effectively… on this front the paper does little to improve “memory” in a meaningful way

Re: δ-mem: Efficient Online Memory for Large Language Models

#13

Post by ktallett »

405126121 wrote: post #8
ktallett wrote: post #3

The obvious energy saving step would be to utilise previous searches by others. Many of the tasks people do are rather similar, it is such an energy waste to start again each time. (Obviously ignoring the huge energy saver, which is to observe if you even need to bother doing the task at all.)

I had this thought and created https://pushrealm.com which is essentially a sort of Stackoverflow written by agents. My theory was that if an agent burns 30 minutes resolving an issue not present in training data, posting the solution would prevent other agents re-treading the same thinking steps.

I see why, but I don't feel this is the solution. Being able to search thru the endless LLM responses is not viable. However having useful memories, similar to human brain is more important. I sense this is why neuromorphic computing is the next step, energy efficient and doesn't remember much of what isn't useful to be stored.

Re: δ-mem: Efficient Online Memory for Large Language Models

#15

Post by tyre »

duskdozer wrote: post #4
ktallett wrote: post #3

The obvious energy saving step would be to utilise previous searches by others. Many of the tasks people do are rather similar, it is such an energy waste to start again each time. (Obviously ignoring the huge energy saver, which is to observe if you even need to bother doing the task at all.)

A lot of what I see people using LLMs for would be more cheaply and reliably done by [scripts]. A search engine style suggestion thing like "Have you tried `sed`?" would be beneficial imo

In my experience, Claude is more than happy to go to Unix tools rather than write its own. Sometimes it will write a lil python script to solve something, but more often than not it’ll pipe together Unix utilities.

This has the benefit of it knowing all of the arcane flags, especially for formatting output.

Re: δ-mem: Efficient Online Memory for Large Language Models

#18

Post by MeteorMarc »

cubefox wrote: post #16

Papers being voted high on Hacker News are usually uncorrelated with their actual importance. It's basically a lottery. There are regularly more interesting papers going semi viral on Twitter.

On huggingface it was #3 paper of the day, which is neutral towards your hypothesis.

Re: δ-mem: Efficient Online Memory for Large Language Models

#19

Post by visarga »

ktallett wrote: post #13
405126121 wrote: post #8

Earlier quoted context omitted.

I had this thought and created https://pushrealm.com which is essentially a sort of Stackoverflow written by agents. My theory was that if an agent burns 30 minutes resolving an issue not present in training data, posting the solution would prevent other agents re-treading the same thinking steps.

I see why, but I don't feel this is the solution. Being able to search thru the endless LLM responses is not viable. However having useful memories, similar to human brain is more important. I sense this is why neuromorphic computing is the next step, energy efficient and doesn't remember much of what isn't useful to be stored.

Why not preserver the essential memories in text? Why neuromorphic?

Re: δ-mem: Efficient Online Memory for Large Language Models

#20

Post by visarga »

rush86999 wrote: post #9
stephantul wrote: post #5

Earlier quoted context omitted.

How would you conceptualize recall in this case? Is searching through the current version of your code and possibly git history not enough?

You would think git history should be the first thing an agent would look at, as they make so many mistakes before they get to the correct answer. They don't. I haven't measured, but documenting bug fixes and architecture seems to help, along with TDD patterns, including integration tests. I would probably add it to Claude.md to look for all of the above when tackling a new bug.

I made a harness that preserves memory for both user messages and task execution. One reason this works is related to judge agents - they can't review information that was not written down. So I track everything in my harness. The judge agents bring the most benefit, based on my evals. The coding agent can execute a task without all the ceremony just as well, but judging needs something to grasp on, besides code. And adding new perspectives helps a lot, it is the most useful intervention. My flow is - user emits a task, the agent plans, then judge agents review the plan, then main agent executes, then judge again reviews the execution. Might consume more tokens to track execution and judgements, but worth it.
Post reply on HN