MemGPT – LLMs with self-editing memory for unbounded context

MemGPT – LLMs with self-editing memory for unbounded context
github.com

Briefly: we tell ChatGPT what API based tools we have, explaines them in 1 sentence and where it can reach their documentation. We added documentations as endpoint. example.com/docs/main is always the starting point that returns high level overview of the app and all available endpoints to call. Every endpoint has its own documentation as well. E.g.: /geocoder has /docs/geocoder documentation endpoint that describes what it does, what input it expects and what it will return.

We also provieded ChatGPT with actions like read_docs, call_endpoint and end_conversation. An action is a structured JSON object with a set of parameters. If ChatGPT wants to interact with the mentioned resources, it emits an action, it gets executed and the answer fed back to it.

With this I can do a task like: "Get a 30 minutes drivetime polygon around 15 Bond Street, London and send it to Foster."

It plans and executes the following all alone. First it calls the geocoder to get the coordinates for the isochrone endpoint, then gets the isochrone by calling the isochrone endpoint and saves it, calls Microsoft Graph API and queries my top 50 connections to find out who Foster is and calls the MS Graph API's send mail endpoint to send the email with attachment to Foster.

It can hierarchically explore the available resources so we don't need a huge context window and we don't have to train the model either. Also we could implement multiple agents. 1 would be a manager and there could me multiple agents to perform each task and return the results to the manager. It would furthet reduce reduce the required context window.

Very likely some BS app will win the Hackathon like always like a market price predictor using Weka's multilayer perceptron with default settings but we believe our solution could be extremely powerful.

I do think this will be way less than having all of the functions listed to begin with though. I think the discoverability is a novel approach. Honestly, I'm surprised ChatGPT with plugins doesn't do something like this by default rather than making you pick which plugins you want at the beginning of the conversation.

Also, we have not implemented it, we can have one or multiple level of managers just like at a company and each would delegate a task to a worker (who could also be a manager) and they would report back the result. Just like in real life, a manager doesn't have to know how something is done, it should only know it's done and the get the results.

We work for a large company and very likely have 100s of apps. We could build wrappers around them e.g. using selenium and we could interact with even old apps.

We could also do the same approach with databases. The db itself would have docs, each table and each field as well. So we could ask ChatGPT to query data from the db and it could fully understand the data before writing the sql query.