41–50 of 59 posts
I also found pure RAG with vector search to not work. I was creating a bot that could find answers to questions about things by looking at Slack discussions. At first, I downloaded entire channels, loaded them into a vector DB, and did RAG. The results sucked. Vector searches don't understand things very well, and in this world, specific keywords and error messages are very searchable. Instead, I take the user's quer…
Earlier quoted context omitted.
re: legal, I saw a post on this idea where their RAG system was designed to return the actual text from the document rather than a LLM response or summary. The LLM played a role in turning the query into the search params, but the insight was that for certain kinds of documents, you want the actual source because of the existing, human written summary or the detailed nuances therein
Sounds more like Generation Augmented Retrieval in that case.
I also found pure RAG with vector search to not work. I was creating a bot that could find answers to questions about things by looking at Slack discussions. At first, I downloaded entire channels, loaded them into a vector DB, and did RAG. The results sucked. Vector searches don't understand things very well, and in this world, specific keywords and error messages are very searchable. Instead, I take the user's quer…
For decades we had search engines based on the query terms (keywords). Then there were lots of discussions and some implementations to put a semantic search on top of it to improve the keyword search. A hybrid search. Google Search did exactly that already in 2015 [1].
Now we start from pure semantic search and put keyword search on top of it to improve the semantic search and call it hybrid search.
In both approaches, the overall search performance is exactly identical - to the last digit.
I am glad, that so far, no one has called this an innovation. But you could certainly write a lot of blog articles about it.
[1] https://searchengineland.com/semantic-search-entity-based-se...
The problem is that most people don't have experience optimizing even 1 of the retrieval systems (vector or keyword), so a lot of users that try to DIY build end up with an awful time trying to get to prod. People are talking about things like RRF (which are needed) but then missing other big-picture things like the mistakes everyone makes when building out a keyword search (not getting the right language rules in place) and also not getting the right vector side (finding the right embedding models, chunking strategies, etc).
I recognize I have a bit of a conflict of interest since I'm at a RAG vendor, but I'll abstain from the name/self-promotion and say: I've seen so many cases where people get this wrong, if you're thinking RAG you really should be hiring a consultant or looking at a complete platform from people that have done it more. Or be prepared to spend a lot of cycles learning and iterating
Having worked building out a RAG SaaS platform for the past year and having worked on the vendor side of several keyword-based search systems in the past 10 years, I can say it's absolutely necessary to have some kind of hybrid search for most use cases I've seen. The problem is that most people don't have experience optimizing even 1 of the retrieval systems (vector or keyword), so a lot of users that try to DIY bui…
Don't think it's overly self-promotional if first asked :)
If you still don't wanna say, feel free to email, email in profile
Having worked building out a RAG SaaS platform for the past year and having worked on the vendor side of several keyword-based search systems in the past 10 years, I can say it's absolutely necessary to have some kind of hybrid search for most use cases I've seen. The problem is that most people don't have experience optimizing even 1 of the retrieval systems (vector or keyword), so a lot of users that try to DIY bui…
As someone who has spent way too long building a RAG system for internal use, would be interested to know what your platform is. Don't think it's overly self-promotional if first asked :) If you still don't wanna say, feel free to email, email in profile
Having worked building out a RAG SaaS platform for the past year and having worked on the vendor side of several keyword-based search systems in the past 10 years, I can say it's absolutely necessary to have some kind of hybrid search for most use cases I've seen. The problem is that most people don't have experience optimizing even 1 of the retrieval systems (vector or keyword), so a lot of users that try to DIY bui…
As someone who has spent way too long building a RAG system for internal use, would be interested to know what your platform is. Don't think it's overly self-promotional if first asked :) If you still don't wanna say, feel free to email, email in profile
I also found pure RAG with vector search to not work. I was creating a bot that could find answers to questions about things by looking at Slack discussions. At first, I downloaded entire channels, loaded them into a vector DB, and did RAG. The results sucked. Vector searches don't understand things very well, and in this world, specific keywords and error messages are very searchable. Instead, I take the user's quer…
I find these discussions funny. For decades we had search engines based on the query terms (keywords). Then there were lots of discussions and some implementations to put a semantic search on top of it to improve the keyword search. A hybrid search. Google Search did exactly that already in 2015 [1]. Now we start from pure semantic search and put keyword search on top of it to improve the semantic search and call it…