Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

MemGPT – LLMs with self-editing memory for unbounded context

github.com

Post reply on HN
Print view View original

11–20 of 90 posts

Re: MemGPT – LLMs with self-editing memory for unbounded context

#11

Post by majewsky »

Aeolos wrote: post #10
o11c wrote: post #2

I don't remember the name, but there's already an Esolang that executes its commands unreliably. Through careful program design, you can ensure that a sequence of commands will execute with 99%, 99.9%, etc. reliability.

Java2000 iirc. 2 decades later, the same approach was unironically popularized for infrastructure as “chaos engineering”.

Not sure if that was /s or not, but it is indeed an important insight to realize that no IT system can have 100% reliability once you factor in hardware failures and power outages. And that's before we talk about bugs.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#12

Post by pacjam »

Update - we just released a Discord perpetual chatbot implemented on top of MemGPT, you can try it here: https://discord.gg/9GEQrxmVyE

You can also run the chatbot demo + a doc QA bot demo (where you can ask MemGPT about API docs) locally with the code on GitHub.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#13

Post by quickthrower2 »

Been thinking about this. My naive way to do this using existing LLMs is to have a JSON the LLM can spit out when it wants to store and then then calling code shoves that in a “RAG” DB. It also has the ability to generate queries if it’s knowledge.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#14

Post by quickthrower2 »

pacjam wrote: post #12

Update - we just released a Discord perpetual chatbot implemented on top of MemGPT, you can try it here: https://discord.gg/9GEQrxmVyE You can also run the chatbot demo + a doc QA bot demo (where you can ask MemGPT about API docs) locally with the code on GitHub.

Thanks! How do I use the bot?

Re: MemGPT – LLMs with self-editing memory for unbounded context

#15

Post by pacjam »

quickthrower2 wrote: post #14
pacjam wrote: post #12

Update - we just released a Discord perpetual chatbot implemented on top of MemGPT, you can try it here: https://discord.gg/9GEQrxmVyE You can also run the chatbot demo + a doc QA bot demo (where you can ask MemGPT about API docs) locally with the code on GitHub.

Thanks! How do I use the bot?

If you want to try the MemGPT Discord chatbot, join the Discord server (linked above), then check out the #memgpt channel to start messaging the bot.

If you want to run the chatbot or API docs examples locally, you can follow the instructions here: https://github.com/cpacker/MemGPT/tree/main#quick-setup.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#16

Post by ansc »

majewsky wrote: post #11
Aeolos wrote: post #10

Earlier quoted context omitted.

Java2000 iirc. 2 decades later, the same approach was unironically popularized for infrastructure as “chaos engineering”.

Not sure if that was /s or not, but it is indeed an important insight to realize that no IT system can have 100% reliability once you factor in hardware failures and power outages. And that's before we talk about bugs.

Not to even mention the heat death of the universe!

Re: MemGPT – LLMs with self-editing memory for unbounded context

#17

Post by pacjam »

quickthrower2 wrote: post #13

Been thinking about this. My naive way to do this using existing LLMs is to have a JSON the LLM can spit out when it wants to store and then then calling code shoves that in a “RAG” DB. It also has the ability to generate queries if it’s knowledge.

This is similar to how our external context is implemented under the hood - you might be interested in our perpetual chat bot example in the GitHub repo (https://github.com/cpacker/MemGPT), the message traces in the demo GIF should give you an idea of how things work at a glance.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#19

Post by empath-nirvana »

Is there any reason you're just doing everything within a single context window? I experimented with similar stuff months ago and basically parallelized everything into multiple requests to different agents in pre and post-processing steps. The main context window, for example, wasn't aware of memories being generated or retrieved. I had a post-processor just automatically generating memories and saving them, along with all the conversations being saved in a vector database, and a pre-processor that would automatically inject relevant memories and context based on the conversation, even re-writing the history so it would look to the main context window like the memory had always been there.

It saved a lot of space in the main context window for unnecessary system prompts and so on.

Re: MemGPT – LLMs with self-editing memory for unbounded context

#20

Post by huevosabio »

Good job!

On the limitations you wrote: ``` Similarly, we also found that the most popular the Llama 2 70B model variants (even those fine-tuned for function calling) would consistently generate incorrect function calls or even hallucinate functions outside the providede schema. ```

You could use grammar-based sampling [0] to ensure that the function call is at least syntactically correct.

[0] https://github.com/ggerganov/llama.cpp/tree/master/grammars

Post reply on HN