Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
Why AutoGPT engineers ditched vector databases
11–20 of 59 posts
Re: Why AutoGPT engineers ditched vector databases
#12Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
Vector DBs don't create embeddings; they store them. As the article points out, the LLM's slowness to respond diminishes the performance that vector DB's can potentially add.
Re: Why AutoGPT engineers ditched vector databases
#13Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
PostgreSQL will match your requirements. Although you won’t load that fast, just because generating the embeddings take longer than that, independently of your storage engine.
Re: Why AutoGPT engineers ditched vector databases
#14« AutoGPT engineers » seem to also generate their articles with LLM, making their documentation awful to grok. For instance, after showing 2 commands, we have to suffer this:
Forge your future! The forge is your innovation lab. All the boilerplate code is already handled, letting you channel all your creativity into building a revolutionary agent. It's more than a starting point, it's a launchpad for your ideas.
In our exploration today, we’ve covered the essentials of working with AutoGPT projects. We began by laying out the groundwork, ensuring you have all the right tools in place. From there, we delved into the specifics of building an effective AutoGPT agent. Trust me, with the right steps, it becomes a straightforward process.
The doc is littered of those paragraphs. Remove the fat! Go to the point! YOLO, that’s a freaking waste of life cycles!
Re: Why AutoGPT engineers ditched vector databases
#15Wouldn't a vector DB be nice to have, so you can use it directly for search?
I understand the argumentation of the article. But I can imagine that waiting so long for a LLM to react that I would actually prefer to do a search instead on a vector database on my "additional information layer" and find relevant information myself. In that case, having a vector DB would then serve two purposes and that could change the considerations whether it is worth the added complexity.
Not an expert here, just a question that came to mind - it might be based on wrong assumptions.
Re: Why AutoGPT engineers ditched vector databases
#16Summary: They stopped using vector databases because the performance benefit simply didn't matter compared to how long the LLMs took to respond, and you should focus on using technology to solve problems and not pick the trendy option. But that never got anyone promoted.
You'd be surprised... I've made a pretty good career of identifying cases where simpler, more efficient designs produce better or equivalent results.
Re: Why AutoGPT engineers ditched vector databases
#17Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
Vector DBs don't create embeddings; they store them. As the article points out, the LLM's slowness to respond diminishes the performance that vector DB's can potentially add.
You can easily create embeddings locally though, with small (L)LMs. Three lines of code using hugging face. I don't understand the point of this article.
Re: Why AutoGPT engineers ditched vector databases
#18They are still using something like a vector DB when it is appropriate. It's just a very simple version built in to the system.
In a just world, this would be added to the title before I clicked the link.
Re: Why AutoGPT engineers ditched vector databases
#19« AutoGPT engineers » seem to also generate their articles with LLM, making their documentation awful to grok. For instance, after showing 2 commands, we have to suffer this: Forge your future! The forge is your innovation lab. All the boilerplate code is already handled, letting you channel all your creativity into building a revolutionary agent. It's more than a starting point, it's a launchpad for your ideas. In o…
I doubt it's generated with an LLM. LLMs are much too easy to get to generate much higher quality text then the one you quoted.
Re: Why AutoGPT engineers ditched vector databases
#20« AutoGPT engineers » seem to also generate their articles with LLM, making their documentation awful to grok. For instance, after showing 2 commands, we have to suffer this: Forge your future! The forge is your innovation lab. All the boilerplate code is already handled, letting you channel all your creativity into building a revolutionary agent. It's more than a starting point, it's a launchpad for your ideas. In o…
I doubt it's generated with an LLM. LLMs are much too easy to get to generate much higher quality text then the one you quoted.
probably generated with a 7B model