Why AutoGPT engineers ditched vector databases
dariuszsemba.com
Why AutoGPT engineers ditched vector databases
1–10 of 59 posts
Re: Why AutoGPT engineers ditched vector databases
#2They are still using something like a vector DB when it is appropriate. It's just a very simple version built in to the system.
Re: Why AutoGPT engineers ditched vector databases
#3I have been working on a system of agents over at https://github.com/agi-merge/waggle-dance - I already split problems up into subtasks for agents to work on independently. I give agents access to vector databases, using a simple global key for now, but soon a context/parent/child key. Access to the vector DBs is proxied via tools (agents have to “call” saveMemory or retrieveMemory). I also check for looping/repetition FREQUENTLY using in-memory vector databases of the langchain agent callback events.
My opinion on this: eh, who cares? AutoGPT and similar are non-standard use cases for Vector DBs right now, and Vector DBs are useful for RAG.
Re: Why AutoGPT engineers ditched vector databases
#4That does make sense for now.
Surely though we’re going to see a fairly exponential increase in these requirements though?
The cheaper the compute gets/scales the more sense it makes to hammer problems with more agents/tries so scale of needed agent memory also goes up.
I’d have gone with “it’s already implemented so just leave it be”
Re: Why AutoGPT engineers ditched vector databases
#5Summary: They stopped using vector databases because the performance benefit simply didn't matter compared to how long the LLMs took to respond, and you should focus on using technology to solve problems and not pick the trendy option.
But that never got anyone promoted.
Re: Why AutoGPT engineers ditched vector databases
#6They are still using something like a vector DB when it is appropriate. It's just a very simple version built in to the system.
Numpy is also my favorite Vector DB.
Re: Why AutoGPT engineers ditched vector databases
#7Is AutoGPT used in a production/productive setting? That would put some perspective on the situations where these insights are applicable.
Re: Why AutoGPT engineers ditched vector databases
#8Where vector databases would make more sense for a project like AutoGPT would be in centralizing distributed memory as a 3rd party service.
If a model on one computer could report memories to a centralized service that could be searched by new instances so work didn't need to be replicated, I'd fully expect that 3rd party service to be running a vector DB.
But in reality, the issues of trust and poisoning the well are too pertinent to see enough centralized consolidation of memory to justify it for a project like this.
I've seen some discussions around E2E encrypted LLM chains, and I could definitely see a 3rd party memory layer as part of that, though I'd suppose it would need to be a plug-in at the model provider and not at the client anyways.
Re: Why AutoGPT engineers ditched vector databases
#9It doesn't have to be a one or the other choice. For example, txtai supports a number of different ANN backends including a simple NumPy implementation (https://neuml.github.io/txtai/embeddings/configuration/ann/). There is value in the plumbing to vectorize data, normalize embeddings and find matches.
It's a good idea to find a solution that enables starting simple and scaling up as needed without having to fully rewrite the code.
Re: Why AutoGPT engineers ditched vector databases
#10I have been working on a system of agents over at https://github.com/agi-merge/waggle-dance - I already split problems up into subtasks for agents to work on independently. I give agents access to vector databases, using a simple global key for now, but soon a context/parent/child key. Access to the vector DBs is proxied via tools (agents have to “call” saveMemory or retrieveMemory). I also check for looping/repetiti…
What's your assessment of the biggest blocking issues for something like this to be practically useful? From what I've seen of AutoGPT things seem to fall apart, in that the goals never quite seem to be achieved once anything more than basic research is requested.