Hybrid might work for English but where are you going to get sparse embeddings like SPLADE or ELSERv2 for most other languages? Vector search with ada-002 or text-003-large capped to the first 500-1000 dimensions will give you a support for 100+ languages for free. If you are using BM25, then you need to train BM25 on every single separate knowledge base which is annoying and expensive.
Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
21–30 of 59 posts
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#22In case you just want a single Postgres function that does RRF (pgvector+fts):
https://supabase.com/docs/guides/ai/hybrid-search
(disclaimer: supabase dev who went down the rabbit hole with hybrid search)
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#23Great article. Hybrid search works well for a lot of scenarios.
The tradeoffs of using existing systems vs building your own resonate with me. What we eventually experienced, however, is that periods of bad search performance often correlated to out-of-date search indices.
I'd be interested in another article detailing how you monitor search. It can be tricky to keep an entire search system moving.
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#24Meilisearch has a really clean implementation of this. Can easily adjust keyword vs vector weighting per query.
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#25RRF is alright, but I've had better results with relative score, or distribution-based scoring.
LlamaIndex has a module for exactly this
https://docs.llamaindex.ai/en/stable/examples/retrievers/rel...
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#26Any tips on accomplishing this in Postgres with pg_vector?
I commented above with a pgvector example that does it-
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#271. Does anyone know a postgres reranking extension, to go beyond RRF through ML models or at least custom code? 2. If anyone is observing significant gains from incorporating knowledge graphs into the retrieval step, what kind of a knowledge graph are you working with, what is your retrieval algorithm, and what technology are you using to store it?
Re 1) pgvector has an example in the repo that uses a model for re-ranking:
https://github.com/pgvector/pgvector-python/blob/master/exam...
I'm not using that in my own experiments since I don't want to worry about the performance of running a model on production, but seems worth a try.
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#28I also found pure RAG with vector search to not work. I was creating a bot that could find answers to questions about things by looking at Slack discussions.
At first, I downloaded entire channels, loaded them into a vector DB, and did RAG. The results sucked. Vector searches don't understand things very well, and in this world, specific keywords and error messages are very searchable.
Instead, I take the user's query, ask an LLM (Claude / Bedrock) to find keywords, then search Slack using the API, get results, and use an LLM to filter for discussions that are relevant, then summarize them all in a response.
This is slow, of course, so it's very multi-threaded. A typical response will be within 30 seconds.
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#29Any tips on accomplishing this in Postgres with pg_vector?
Supabase has some good examples on their website, search for hybrid search. I needed to tune the function they have there but it should show you how to approach it.
Here is our doc with RRF:
Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search
#30Earlier quoted context omitted.
Author here: you're for sure right -- it's not a problem with RAG the theoretical concept. In fact, I think RAG implementations should likely be specific to their use cases (e.g. our hybrid search approach works well for customer support, but I'm not sure if it would work as well in other contexts, say for legal bots). I've seen the whole gamut of RAG implementations as well, and the implementation, specifically prom…
re: legal, I saw a post on this idea where their RAG system was designed to return the actual text from the document rather than a LLM response or summary. The LLM played a role in turning the query into the search params, but the insight was that for certain kinds of documents, you want the actual source because of the existing, human written summary or the detailed nuances therein
Sounds more like Generation Augmented Retrieval in that case.