Very little of this is RAG but rather just FTS with clever reformulation and re-ranking. RAG is about providing an grounded response, given the actual data in the corpus. Great article and content, nonetheless!!
author here - thank you!
RAG Is Simpler Than You Think
141–150 of 166 posts
Re: RAG Is Simpler Than You Think
#142RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector search as magic pixie dust that makes search great without effort is not necessarily going to work that well. Also, it can add a lot of cost and complexity to the equation. And if not tuned properly, you don't necessarily get good results. The key thing with R…
> With search, and by extension RAG, the principle of shit in, shit out applies
Similar to SEO on marketing pages, we started rewriting product docs around the idea that it will be consumed by a RAG. Mostly by putting a lot of focus on well structured headlines, thinking more carefully about technical terminology vs common human-language questions, occasionally using variations of keywords in the text, etc. This applies to pure LLM consumption too, not just hybrid search.
Once you start tracking what users are asking you learn to adapt the documentation around it. And LLMs can also suggest improvements by comparing questions vs search results vs LLM responses.
Re: RAG Is Simpler Than You Think
#143I would like to see how each recipe performs against its corresponding evals. Some sort of ranking would be useful. Everyone keeps posting articles about how to implement RAG, but I also wonder why there isn’t some sort of skill to help people create a simple retrieval plan, starting with the retrieval methods and connecting them with evals. This could show whether they actually improve the result and make retrieval…
author here - that's an amazing idea. would be an insanely large article though - maybe will write up a series
Re: RAG Is Simpler Than You Think
#144Earlier quoted context omitted.
I thought text search was always the first thing you try, then fuzzy search, then you go for RAG
I think Bitwarden implemented some vector search in their password search feature ... totally annoying it gives me back all kinds of stuff that I don't care. I want fuzzy search like 95% of time and then I might consider having additional list of things that can be suggested by vector search.
ahh now I realize why I get so much completely irrelevant search results in many sites recently. I mean I'm searching for betel and you're giving me nuts. haha
Re: RAG Is Simpler Than You Think
#145I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…
I thought text search was always the first thing you try, then fuzzy search, then you go for RAG
I wish everyone thought like you, in my experience unfortunately it's not the case
Re: RAG Is Simpler Than You Think
#146Earlier quoted context omitted.
Maybe if a person can't even google RAG they are not the intended audience of that article.
When I hear stuff like this I always imagine going to a restaurant and asking the waitress for a menu and them replying “lol just google it”. It’s not that I can’t or don’t know how, it’s rather that the expectation should be that a website should… link you to the information it believes to be relevant background. It’s why it’s called a “web”, linking is a core concept.
given the audience and the venue I think it's more like going to a restaurant as a customer and then asking the waiter to explain what a sandwich is.
Re: RAG Is Simpler Than You Think
#147I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…
I thought text search was always the first thing you try, then fuzzy search, then you go for RAG
Even that is an oversimplification unless you are doing something very basic.
Volume of documents, size of documents, versioning, frequency of update, documents similar or overlapping information, how much or exactly what you need for the LLM to understand, AI friendly documents, who has access and at what level, blue teaming, red teaming, multi-lingual, does the LLM know the domain language of the user and documents.
I probably missed a few things even with that.
Re: RAG Is Simpler Than You Think
#148Earlier quoted context omitted.
> When I hear stuff like this I always imagine going to a restaurant and asking the waitress for a menu and them replying “lol just google it”. in this case there was a menu in the next empty table and you saw it but in place of getting it you want the waitress to get it for you. Which is a normal behavior but you could save your time by just getting the menu yourself.
No. It isn't. With acronyms, there's often plenty of potential things it can stand for, and if the person doesn't know enough to know which one is the correct acronym, Googling it isn't going to help them. As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK." The NWS knows this and automatically links to their glossary for both acronyms as well as jargon…
>As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK."
it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing.
if you're reading this stuff, and you can't figure out what kind of RAG that the search engine mentioned is being talked about through context clues, or you aren't clever enough to feed context into the search like 'hackers , computers, rag' as a query -- there is a very high probability that the person will have absolutely nothing constructive to add to the conversation that is about the topics they haven't even yet conceptualized or are aware of.
in that case that slight barrier to entry for the conversation will serve as a tool to produce less work for the moderators and derail less threads into uselessness.
(much like this stupid divergence.)
Re: RAG Is Simpler Than You Think
#149I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…
RAG only makes sense if you have an LLM review the results, pick the most relevant ones and iterate further if there's a need running another query and repeating the process. Raw dump of vector search (even with reranking) is asking for troubles (or rather weird user questions like 'why this crap popped up in the results?')
Re: RAG Is Simpler Than You Think
#150I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…
Re: the rube goldberg machine of diminishing returns https://www.anthropic.com/engineering/contextual-retrieval This is from two years ago, but I think it's still SotA?
That is the approach I would take today. Late Interaction is worth a look. Evals are necessary.