41–50 of 59 posts
Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
Generating the embeddings is by far the slowest part of that, but it’s embarrassingly parallel so if you have $ it can be done that quick. When I worked for Dubai airport, I was tasked with building a vector similarity search that was highly optimised for query speed, in the end I ended up holding the vectors in memory (in a numpy array) and using scipy to do cosine similarity, I could get about 1.2 million vectors p…
Summary: They stopped using vector databases because the performance benefit simply didn't matter compared to how long the LLMs took to respond, and you should focus on using technology to solve problems and not pick the trendy option. But that never got anyone promoted.
Has anyone ever managed to generate anything useful with AutoGPT? I had several attempts and apart from wasting some money for GPT-4 API calls, it's never produced anything usable. Whereas if I manually enter prompts in ChatGPT I can often produce simpler project from the beginning till the end, if I partition them into logically independent parts.
Earlier quoted context omitted.
Don't read the intermediate representation. The idea is that you use an LLM to summarize those comments into human readable text.
aw geez, now we have to compile docs, too?!
--Khrushchev, satirically inquiring regarding the implausible number of labor-saving devices exhibited on the American side of the Kitchen Debate[1].
They are still using something like a vector DB when it is appropriate. It's just a very simple version built in to the system.
Is there any implementation of open source vector db that is fast enough to say create embedding of 100M documents locally within few hours and find ranked matches in under a second? I tried ChromaDb and it is super slow, basically unusable.
I do about 100 million embeddings using around 50 GPU instances and feed them into Qdrant. Takes about 12 hours. Very happy with the result and performance as long as you have the option to have a very large memory instance running.
Earlier quoted context omitted.
What's your assessment of the biggest blocking issues for something like this to be practically useful? From what I've seen of AutoGPT things seem to fall apart, in that the goals never quite seem to be achieved once anything more than basic research is requested.
It can do stuff, sort of, like helping me to create ; https://github.com/tudorw/Ai_MegaList/blob/main/AI_Sector_Br... After a lot of trial and error, I managed to keep it somewhat on track by using a CSV file, something like; "An expert manipulate .csv files, read the first URL from the first line, 2nd column of 'raw.csv', pass the URL to browse_website the questions 'summarize the activities, highlight any investmen…
They are still using something like a vector DB when it is appropriate. It's just a very simple version built in to the system.