Agentic Context Management: Memory and Cost as Architecture Problems
1–10 of 42 posts
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#2I also wrote a shorter preview here: https://www.maximem.ai/blog/agentic-context-management-paper
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#3I like to start with memory engineering then reach full system then reducing costs. This allows unlocking full potential of agents.
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#4ACM, that's the term that I'd been looking for - and your paper explains it clearly. At the end, most of LLM problems are context problems. Getting the correct knowledge into its context window without overpopulating it is the actual engineering effort for most agents. And the solution you present seems promising.
Both compaction with validation and predictive fetching are the way to go.
I do not want to write an implementation for this myself, and if Synap is that implementation, I'd like to ask you a few questions: 1. Does it work with context that's not just agent conversations, but rather documents? 2. Is it better than RAG on large dataset? 3. What does on-prem options look like?
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#5I also wrote a shorter preview here: https://www.maximem.ai/blog/agentic-context-management-paper
your website isnt working
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#6ACM, that's the term that I'd been looking for - and your paper explains it clearly. At the end, most of LLM problems are context problems. Getting the correct knowledge into its context window without overpopulating it is the actual engineering effort for most agents. And the solution you present seems promising. Both compaction with validation and predictive fetching are the way to go. I do not want to write an imp…
[flagged]
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#7[flagged]
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#8Context drift on retries is easily the most annoying part of this setup. Locking down the tool payload schema first was the only thing that worked for us
Re: Agentic Context Management: Memory and Cost as Architecture Problems
#9Context pollution and rot are probably more important than memory, because facts can usually be retrieved if the agent is good at following breadcrumbs.
What's also the biggest killer is code rot. Agents are particularly good at death by thousand cuts. They implement something poorly, or incorrectly, or introduce a bad pattern into the project. Then they continue to amplify that badness over time, as they continue to copy from it on subsequent work. It spreads like a virus.
Keeping these seeds out of the project is very difficult, and cleaning up the rot is very difficult. It also seems like a hard problem to solve because following the existing codebase is something that is good when the code is good, but bad when it is bad. So, seemingly, the solution means more thinking and evaluation for every change that is being made.