Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MemGPT – LLMs with self-editing memory for unbounded context

github.com

Post reply on HN
Print view View original

21–30 of 90 posts

Re: MemGPT – LLMs with self-editing memory for unbounded context

#21

Post by spaintech »

empath-nirvana wrote: post #19

Is there any reason you're just doing everything within a single context window? I experimented with similar stuff months ago and basically parallelized everything into multiple requests to different agents in pre and post-processing steps. The main context window, for example, wasn't aware of memories being generated or retrieved. I had a post-processor just automatically generating memories and saving them, along w…

This is a fascinating approach. I’m working on something similar but as part of the feedback loop, as you said, rewriting history with transactional data as part of the context window. I feel as though the LLM and the NLP could potentially be a more realizable interface to structured data, well, I should say, this is the idea we are exploring. For us, as data is created (within a certain context of the business) we extract the data, generate the embeddings and build out the vector database as to:

Pre and Post-Processing:

- Post-Processing: After the main model responds, a post-processor takes over, automatically generating memories from the conversation and saving them. This ensures that important context is stored without burdening the primary model with these tasks. We also execute any relevan business logic as part of the request, then feed that back to the systems…

- Pre-Processing: Before a new input is sent to the main model, a pre-processor checks saved memories and injects relevant context. * executes logic * It’s as if this pre-processor gives the main model a “refresher” on prior conversations, preparing it to provide more informed and consistent responses.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#22

Post by keithnoizu »

empath-nirvana wrote: post #19

Is there any reason you're just doing everything within a single context window? I experimented with similar stuff months ago and basically parallelized everything into multiple requests to different agents in pre and post-processing steps. The main context window, for example, wasn't aware of memories being generated or retrieved. I had a post-processor just automatically generating memories and saving them, along w…

Yes, I have a similar solution.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#25

Post by Difwif »

I've had a suspicion for a while now that this is what ChatGPT does within a conversation (chat.openai.com, not the api). I've had very long chat histories that seem to gracefully degrade instead of just forgetting everything. Maybe there's more clues in the context than I realize though.

Either way this type of idea will probably be a fundamental feature for all chat bots in the future IMO.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#27

Post by minimaxir »

swyx wrote: post #26
qup wrote: post #24

Discussed last night: https://news.ycombinator.com/item?id=37894403 (Mostly arguing about the authors' choice of title)

[flagged]

It's a necessary marketing tactic lately to surface AI projects above the bullshit hype. The AI space hypesters who do podcasts and speak at conferences do not care about cringe.

The real deterrent to adding GPT to a project name is a cease-and-desist from OpenAI.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#28

Post by Der_Einzige »

Difwif wrote: post #25

I've had a suspicion for a while now that this is what ChatGPT does within a conversation (chat.openai.com, not the api). I've had very long chat histories that seem to gracefully degrade instead of just forgetting everything. Maybe there's more clues in the context than I realize though. Either way this type of idea will probably be a fundamental feature for all chat bots in the future IMO.

ChatGPT degrades precisely because they aren't doing anything special to extend their memory beyond the context length.

There are trivial techniques to implement "lossy" memory, such as just average pooling tokens (the same approach used by sentence transformers). Not sure why it's so rare to see this used for condensing a huge amount of context into a prompt. It is effectively "medium" term memory.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#29

Post by cosmojg »

swyx wrote: post #26
qup wrote: post #24

Discussed last night: https://news.ycombinator.com/item?id=37894403 (Mostly arguing about the authors' choice of title)

[flagged]

I mean, if they're building on top of what is commonly understood as a generative pre-trained transformer model[1], then that just seems accurate? Although, I do agree that it's pretty cringe when they do it purely as a deceptive marketing tactic rather than to communicate meaningful information.

[1] https://en.m.wikipedia.org/wiki/Generative_pre-trained_trans...

Post reply on HN