Instead of embedding the user prompt, I let the LLM invert it into keywords and search the embedding of that. It very much does feel like a magic bullet.
"Search the embedding"? Could you elaborate on this, it sounds interesting!
Build a search engine, not a vector DB
11–20 of 84 posts
Re: Build a search engine, not a vector DB
#12I have a related project here: https://findsight.ai and also gave a talk about building it here: https://youtu.be/elNrRU12xRc
Re: Build a search engine, not a vector DB
#13Instead of embedding the user prompt, I let the LLM invert it into keywords and search the embedding of that. It very much does feel like a magic bullet.
"Search the embedding"? Could you elaborate on this, it sounds interesting!
Ask the LLM to summarize the question, then take an embedding of that.
I think you can do the same with data you store… summarize it to same number of tokens, then get an embedding for that to save with the original text.
Test! Different combinations of summarizing LLM and embedding generation LLM can get different results. But once you decide, you are locked in the summarizer as much as the embedding generator.
Not sure is this is what the parent meant though.
Re: Build a search engine, not a vector DB
#14Instead of embedding the user prompt, I let the LLM invert it into keywords and search the embedding of that. It very much does feel like a magic bullet.
"Search the embedding"? Could you elaborate on this, it sounds interesting!
I think OP means to filter the user input through an LLM with “convert this question into a keyword list” and then calculating the embedding of the LLM’s output (instead of calculating the embedding of the user input directly). The “search the embedding” is the normal vector DB part.
Re: Build a search engine, not a vector DB
#15Earlier quoted context omitted.
"Search the embedding"? Could you elaborate on this, it sounds interesting!
Ask the LLM to summarize the question, then take an embedding of that. I think you can do the same with data you store… summarize it to same number of tokens, then get an embedding for that to save with the original text. Test! Different combinations of summarizing LLM and embedding generation LLM can get different results. But once you decide, you are locked in the summarizer as much as the embedding generator. Not…
BTW: I think of this like asking someone to put things into their own words, and then it’s easier for them to remember. Matching on your way of talking can be weird from the LLM’s point of view, so use their point of view!
Re: Build a search engine, not a vector DB
#16I agree too. My impression is that almost all RAG tutorials _only_ talk about vector DBs, when these are not strictly required for Retrieval Augmented Generation. I'm guessing vector DBs are useful when you have massive amounts of documents on diverse topics. Some gotchas I experienced (but I might be using the wrong embedding/vector DB: spaCy/FAISS): - Short user questions might result a low signal query vector, e.…
Lucene supports decompounding and stemming, https://core.ac.uk/reader/154370300 depending on the language decompounding can be very important or of little import, Germanic languages should probably have decompounding.
Re: Build a search engine, not a vector DB
#17Many, many big companies don't see any value in search. They simply use the defaults, and when those defaults are abysmal (like in the case of Confluence for example), well... they just suffer through it in silence.
I have so far mostly failed in trying to explain 1/ why search matters and 2/ that not all "search" functionality are equal and that building good search is an art form.
Re: Build a search engine, not a vector DB
#18I agree too. My impression is that almost all RAG tutorials _only_ talk about vector DBs, when these are not strictly required for Retrieval Augmented Generation. I'm guessing vector DBs are useful when you have massive amounts of documents on diverse topics. Some gotchas I experienced (but I might be using the wrong embedding/vector DB: spaCy/FAISS): - Short user questions might result a low signal query vector, e.…
Lucene supports decompounding and stemming, https://core.ac.uk/reader/154370300 depending on the language decompounding can be very important or of little import, Germanic languages should probably have decompounding.
I wonder what the advantage/disadvantages of dedicated search tools like lucent instead of custom LLC.
Re: Build a search engine, not a vector DB
#19I agree too. My impression is that almost all RAG tutorials _only_ talk about vector DBs, when these are not strictly required for Retrieval Augmented Generation. I'm guessing vector DBs are useful when you have massive amounts of documents on diverse topics. Some gotchas I experienced (but I might be using the wrong embedding/vector DB: spaCy/FAISS): - Short user questions might result a low signal query vector, e.…
Neo4j are mixing vector embeddings with knowledge graphs - https://neo4j.com/generativeai/
Re: Build a search engine, not a vector DB
#20From the article: "The crux is that while vector search is better along some axes than traditional search, it's not magic. Just like regular search, you'll end up with irrelevant or missing documents in your results."
RAG is often helpful and easy to add, but it's fundamentally search - not magic.
I find it helpful to look at the search results before feeding them into the model. Just like the "I'm feeling lucky" button on google doesn't always give the perfect answer. You may have to tweak your search query to improve the result.