Has anyone ever managed to generate anything useful with AutoGPT? I had several attempts and apart from wasting some money for GPT-4 API calls, it's never produced anything usable. Whereas if I manually enter prompts in ChatGPT I can often produce simpler project from the beginning till the end, if I partition them into logically independent parts.
Why AutoGPT engineers ditched vector databases
31–40 of 59 posts
Re: Why AutoGPT engineers ditched vector databases
#32Wouldn't a vector DB be nice to have, so you can use it directly for search? I understand the argumentation of the article. But I can imagine that waiting so long for a LLM to react that I would actually prefer to do a search instead on a vector database on my "additional information layer" and find relevant information myself. In that case, having a vector DB would then serve two purposes and that could change the c…
I think the article is a near miss on the right idea. The important point is that a _dedicated_ vector database is probably overkill and not justified for most real-world use cases.
But a multi-modal database that also supports embeddings in hybrid mode or _in addition_ to standard retrieval techniques is both still very useful, and probably sufficient.
What that means to me is that it is yet another vote in favor of less optimized but far more versatile and robust solutions like: OpenSearch, Elastic, and PostgreSQL. [when I say 'less optimized' I'm only referring to their current vectordb plugins, not the rest of the machinery]
OpenSearch and Postgre are phenonemal, robust, OSS tools and the only lingering downside seems to be that their vectordb implementations are still a bit less optimized for large collections - but that probably doesn't matter in practice.
Re: Why AutoGPT engineers ditched vector databases
#33Wouldn't a vector DB be nice to have, so you can use it directly for search? I understand the argumentation of the article. But I can imagine that waiting so long for a LLM to react that I would actually prefer to do a search instead on a vector database on my "additional information layer" and find relevant information myself. In that case, having a vector DB would then serve two purposes and that could change the c…
The similarity operation is just a dot product, practically a sum over position wise products of two vectors. A for loop with an addition and a multiply inside. That's all you need after embedding. You can use np.dot() to get exact similarity scores and better retrieval with very fast times for under 100K vectors.
You only want the approximate nearest neighbour method for millions of vectors and above. Even that is easy to do with off the shelve libraries for local index. It only gets complicated when you want fast insertion and distributed access.
Re: Why AutoGPT engineers ditched vector databases
#34I was expecting the authors to talk about superficial encoding of information (words and phrases) as opposed to their meaning or implications. For example the text "The general idea behind Q-learning and SARSA" won't embed the same with "reinforcement". Or "Count the letters in this phrase." won't embed the same with "27".
This can cause RAG systems to fail to retrieve, fail to connect fragmented information or to conclude the result of a process. My theory is that information needs to be digested by a LLM and augmented before indexing in a RAG system. Embeddings are just searching at surface level. That is why I thought AutoGPT had difficulties, one of the reasons at least.
Maybe we can have LLMs preprocess the material to expose the deeper layers of meaning. And we need to reindex everything when we discover we are interested in an aspect we didn't explicitly expose for retrieval. Study, then index, and sometimes study again.
Re: Why AutoGPT engineers ditched vector databases
#35Wouldn't a vector DB be nice to have, so you can use it directly for search? I understand the argumentation of the article. But I can imagine that waiting so long for a LLM to react that I would actually prefer to do a search instead on a vector database on my "additional information layer" and find relevant information myself. In that case, having a vector DB would then serve two purposes and that could change the c…
They do still carry out search, but using a brute force approach (dot product over all the embeddings) instead of vector db. Their point is that they aren't likely to generate enough messages for brute force to be an issue. (Even over 100s of thousands of embeddings brute force is pretty fast.)
> Even over 100s of thousands
* number of languages, no?
Re: Why AutoGPT engineers ditched vector databases
#36Wouldn't a vector DB be nice to have, so you can use it directly for search? I understand the argumentation of the article. But I can imagine that waiting so long for a LLM to react that I would actually prefer to do a search instead on a vector database on my "additional information layer" and find relevant information myself. In that case, having a vector DB would then serve two purposes and that could change the c…
The similarity operation is just a dot product, practically a sum over position wise products of two vectors. A for loop with an addition and a multiply inside. That's all you need after embedding. You can use np.dot() to get exact similarity scores and better retrieval with very fast times for under 100K vectors. You only want the approximate nearest neighbour method for millions of vectors and above. Even that is e…
Ok got it now, thanks!
Re: Why AutoGPT engineers ditched vector databases
#37I asked them about this in their discord. they didnt give me straight-forward answers. that’s when I knew I was right from the start that this whole thing was a show
Re: Why AutoGPT engineers ditched vector databases
#38Summary: They stopped using vector databases because the performance benefit simply didn't matter compared to how long the LLMs took to respond, and you should focus on using technology to solve problems and not pick the trendy option. But that never got anyone promoted.
More like hype-driven development and magpies making decisions based on how shiny the bracelet is.
Re: Why AutoGPT engineers ditched vector databases
#39« AutoGPT engineers » seem to also generate their articles with LLM, making their documentation awful to grok. For instance, after showing 2 commands, we have to suffer this: Forge your future! The forge is your innovation lab. All the boilerplate code is already handled, letting you channel all your creativity into building a revolutionary agent. It's more than a starting point, it's a launchpad for your ideas. In o…
Don't read the intermediate representation. The idea is that you use an LLM to summarize those comments into human readable text.
aw geez, now we have to compile docs, too?!
Re: Why AutoGPT engineers ditched vector databases
#40Has anyone ever managed to generate anything useful with AutoGPT? I had several attempts and apart from wasting some money for GPT-4 API calls, it's never produced anything usable. Whereas if I manually enter prompts in ChatGPT I can often produce simpler project from the beginning till the end, if I partition them into logically independent parts.
Unfortunately I've not. For the tasks that I've used these types of tools on I found ChatGPT's "Advanced Data Analysis" mode significantly more useful.