Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

assembled.com

Post reply on HN
Print view View original

31–40 of 59 posts

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#31

Post by esafak »

pamelafox wrote: post #27
esafak wrote: post #12

1. Does anyone know a postgres reranking extension, to go beyond RRF through ML models or at least custom code? 2. If anyone is observing significant gains from incorporating knowledge graphs into the retrieval step, what kind of a knowledge graph are you working with, what is your retrieval algorithm, and what technology are you using to store it?

Re 1) pgvector has an example in the repo that uses a model for re-ranking: https://github.com/pgvector/pgvector-python/blob/master/exam... I'm not using that in my own experiments since I don't want to worry about the performance of running a model on production, but seems worth a try.

That's outside the database, though. This is closer to what I had in mind: https://postgresml.org/blog/how-to-improve-search-results-wi...

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#32

Post by siquick »

thefourthchime wrote: post #28

I also found pure RAG with vector search to not work. I was creating a bot that could find answers to questions about things by looking at Slack discussions. At first, I downloaded entire channels, loaded them into a vector DB, and did RAG. The results sucked. Vector searches don't understand things very well, and in this world, specific keywords and error messages are very searchable. Instead, I take the user's quer…

When you’re creating your embedding you can store keywords from the content (using an LLM) in the metadata of each chunk which would positively increase the relevancy of results turned from the retrieval.

LlamaIndex does this out of the box.

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#33

Post by janalsncm »

Reciprocal rank scoring is just one way of forcing scores into a fixed distribution: in this case, decaying with the reciprocal of its rank. But it also assumes fixed weight from all components, i.e. the top ranked keyword match has equal relevance to the top ranked semantic match.

There are a couple ways around this. Either learning the relative importance based on the query, and/or using a separate reranking function (usually a DNN) that also takes user behavior into account.

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#34

Post by benreesman »

thefourthchime wrote: post #28

I also found pure RAG with vector search to not work. I was creating a bot that could find answers to questions about things by looking at Slack discussions. At first, I downloaded entire channels, loaded them into a vector DB, and did RAG. The results sucked. Vector searches don't understand things very well, and in this world, specific keywords and error messages are very searchable. Instead, I take the user's quer…

Zero shot key phrase extraction is a reasonably well-studied field. I don’t know what the current SOTA is, but the one that was pretty hot shit last time I needed one was kbir-inspec which is on HuggingFace and you can test it right on the page.

Might be worth a shot if performance is a tricky spot in your setup.

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#35

Post by yingfeng »

RRF is a simple and effective means of fused ranking for multiple recall. Within our open source RAG product RAGFlow(https://github.com/infiniflow/ragflow), Elasticsearch is currently used instead of other general vector databases, because it can provide hybrid search right now. Under the default cases, embedding based reranker is not required, just RRF is enough, while even if reranker is used, keywords based retrieval is also a MUST to be hybridized with embedding based retrieval, that's just what RAGFlow's latest 0.7 release has provided.

On the other hand let me introduce another database we developed, Infinity(https://github.com/infiniflow/infinity), which can provide the hybrid search, you can see the performance here(https://github.com/infiniflow/infinity/blob/main/docs/refere...), both vector search and full-text search could perform much faster than other open source alternatives.

From the next version(weeks later), Infinity will also provide more comprehensive hybrid search capabilities, what you have mentioned the 3-way recalls(dense vector, sparse vector, keyword search) could be provided within single request.

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#36

Post by visarga »

edude03 wrote: post #2

Thanks for sharing, I like the approach and it makes a lot of sense for the problem space. Especially using existing products vs building/hosting your own. I was however tripped up by this sentence close to the beginning: > we encountered a significant challenge with RAG: relying solely on vector search (even using both dense and sparse vectors) doesn’t always deliver satisfactory results for certain queries. Not to…

> Not to be overly pedantic, but that's a problem with vector similarity, not RAG as a concept.

Vector similarity has a surprising failure mode. It only indexes explicit information, missing out the implicit one. For example "The second word of this phrase, decremented by one" is "first", do you think these strings will embed the same? Calculated results don't retrieve well. Also, deductions in general.

How about "I agree with what John said, but I'd rather apply Victor's solution"? It won't embed like the answer you seek. Multi-hop information seeking questions don't retrieve well.

The obvious fix is to pre-ingest all the RAG text into a LLM and calculate these deductions before embedding.

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#37

Post by owen-elliott »

The composability of RRF is definitely one of its most appealing characteristics. It doesn't matter what algorithm or vendor you have, you can just fuse with ranks alone. I've seen it shine when fusing lexical and vector search results where semantic attributes like styles and exact attributes like quantities are mixed together in queries, e.g., "modern formal watch with 40mm face".

While it's not such a problem in RAG, one downside is that it complicates pagination for results (there are a few different ways to tackle this).

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#38

Post by cricketlover »

Pardon my ignorance but I was hung up on this line.

> Out-of-sync document stores could lead to subtle bugs, such as a document being present in one store but not another.

But then the article suggests to upload synchronously in S3/DDB and then sync asynchronously to actual document stores. How does this solve out of sync issue? It doesn't. It can't be solved is what I'm thinking.

> Data, numbers

How much data are we talking about?

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#39

Post by verdverm »

gradys wrote: post #30
verdverm wrote: post #5

Earlier quoted context omitted.

re: legal, I saw a post on this idea where their RAG system was designed to return the actual text from the document rather than a LLM response or summary. The LLM played a role in turning the query into the search params, but the insight was that for certain kinds of documents, you want the actual source because of the existing, human written summary or the detailed nuances therein

Sounds more like Generation Augmented Retrieval in that case.

It wasn't this GAR post, I remember them calling out legal docs explicitly, might have seen it on Twitter

https://blog.luk.sh/rag-vs-gar

Re: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search

#40

Post by testfoo444 »

yingfeng wrote: post #35

RRF is a simple and effective means of fused ranking for multiple recall. Within our open source RAG product RAGFlow( https://github.com/infiniflow/ragflow ), Elasticsearch is currently used instead of other general vector databases, because it can provide hybrid search right now. Under the default cases, embedding based reranker is not required, just RRF is enough, while even if reranker is used, keywords based retr…

Elastic Search is publishing a lot of interesting posts on this topic although with a bit of marketing for ex https://www.elastic.co/search-labs/blog/semantic-reranking-w...
Post reply on HN