Build a search engine, not a vector DB
blog.elicit.com
Build a search engine, not a vector DB
1–10 of 84 posts
Re: Build a search engine, not a vector DB
#2Agree fully, vector search in embedding space is insufficient if you are working wirh a single document domain (i.e. They are all fish restaurant menu) and then the only thing that can save you is text search. Just make sure the underlying database supports synonyms lists and normalization in the languages you plan using.
About the "bad news" section.
You can do that today by just asking the llm using the ReAct pattern. Give it the database schema, a few shots prompt, and will happily decide to build query, read titles, and do more query if the titles aren't relevant enough, and fetch the content of titles that are relevant and use those to form an opinion.
This may not sem fast, but there are 7b token models that can do it today, at 150+token/second.
Re: Build a search engine, not a vector DB
#3I'd love to have a search engine for all of my different conversations I've ever had with people through various messaging apps, that combines email and my scanned documents through paperless-ngx and any other PDFs or documents in my nextcloud in a single search interface
Re: Build a search engine, not a vector DB
#4I'd love to have a search engine for all of my different conversations I've ever had with people through various messaging apps, that combines email and my scanned documents through paperless-ngx and any other PDFs or documents in my nextcloud in a single search interface
Maybe at some point the NSA will let us download them all!
Re: Build a search engine, not a vector DB
#5Agree fully, vector search in embedding space is insufficient if you are working wirh a single document domain (i.e. They are all fish restaurant menu) and then the only thing that can save you is text search. Just make sure the underlying database supports synonyms lists and normalization in the languages you plan using. About the "bad news" section. You can do that today by just asking the llm using the ReAct patte…
please elaborate, thanks.
Re: Build a search engine, not a vector DB
#6I'm trying to alleviate the issue with tagging ([link redacted]), but it's not a panacea.
I feel that a big part of the solution will simply be in the form of increased speeds. If you can ask the model for a strategy and then let it search/process a few times in a loop, responses will improve vastly.
Re: Build a search engine, not a vector DB
#7I agree too. My impression is that almost all RAG tutorials _only_ talk about vector DBs, when these are not strictly required for Retrieval Augmented Generation. I'm guessing vector DBs are useful when you have massive amounts of documents on diverse topics.
Some gotchas I experienced (but I might be using the wrong embedding/vector DB: spaCy/FAISS):
- Short user questions might result a low signal query vector, e. g. user : "Who is Keanu Reeves?" -> false positives on Wikipedia articles which only contain "Who is"
- Typos and formatting affects the vectorization, a small difference might lead to a miss, e.g. "Who is Keanu Reeves?" -> match, "Who is keanu Reeves?" -> no match, no match with any other capitalization.
If there's only a single document, a simple keyword search might lead to better results.
In my experience, false positives (retrieving an irrelevant text and generating completely wrong answer) are a bigger problem than negatives (not retrieving text, possibly can't answer question).
Has somebody experience with Apache Lucene / Solr or Elasticsearch?
Re: Build a search engine, not a vector DB
#8I'm trying to alleviate the issue with tagging ([link redacted]), but it's not a panacea. I feel that a big part of the solution will simply be in the form of increased speeds. If you can ask the model for a strategy and then let it search/process a few times in a loop, responses will improve vastly.
I joke that is akin to applying taxonomy on a live tv interview. You need to tag and categorize but may only do so with precision after a point is made.
My current solution is to have an nlp pipeline that does so as tokens are returned. Not quite as precise yet but shows promise.
Should be open source sooner rather than later.
Re: Build a search engine, not a vector DB
#9Instead of embedding the user prompt, I let the LLM invert it into keywords and search the embedding of that. It very much does feel like a magic bullet.
Re: Build a search engine, not a vector DB
#10I'm trying to alleviate the issue with tagging ([link redacted]), but it's not a panacea. I feel that a big part of the solution will simply be in the form of increased speeds. If you can ask the model for a strategy and then let it search/process a few times in a loop, responses will improve vastly.
I joke that is akin to applying taxonomy on a live tv interview. You need to tag and categorize but may only do so with precision after a point is made. My current solution is to have an nlp pipeline that does so as tokens are returned. Not quite as precise yet but shows promise. Should be open source sooner rather than later.
I like that analogy.