171–180 of 197 posts
OT but its interesting that none of the harnesses today use embeddings but just simple grep. I would not have predicted this
cursor still uses embeddings and theyve found it works better than just grep https://cursor.com/blog/semsearch
Earlier quoted context omitted.
As simple as a prompting it with structural output or restrictions for your criteria. With agents, the prompting could be dynamic for maximum accuracy for every retrieval. This absolutely would beat the best of the best embedding-based RAG models. Nobody uses this now mainly due to speed. An llm retrieval would be 10x or more slower than embedding. You can try that now Take some failing cases or bad retrieval from yo…
So uh ... Where's the retrieval part? You know RAG is used to implement that, right? You're basically saying "we don't need an ALU, we can just use the Windows calculator"
I notice a lot of these AI written articles share this pattern where they'll present idea 1, then idea 2, and finally idea 3 which is some amalgamation of idea 1 and 2. Claude especially will present hybrid options and compromises to avoid having to make a choice then framing the hybrid option as the "best of both worlds" when they're borderline nonsensical. "on the fly embedding" and "Sparse + dense reranking" don't…
This has Claude written all over it. "Recipe 4: On-The-Fly Embedding (The Fresh Data Play) The insight If your data changes frequently, why pay to re-embed everything?" This reads like every Claude generated presentation I've seen.
More LLM-generated text about LLMs. Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.
Earlier quoted context omitted.
author here. at most companies I've worked for recently (F500) RAG is still quite trendy. this was what frustrated me a bit and motivated to write this article - along with other experiences that definitely relate with some of the folks in the comments above
what I implied is that in 2024 a lot was written about RAG, LLMs were trained on it, and that explains your gpt-4o-mini reference. the argument is that the article is ai slop, and that point is totally unrelated to name dropping fortune 500s references
I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…
Can you elaborate? We have technicians searching in different languages. Also our knowledge base is often in different languages. I just don't see how full text search can work? Maybe in a problem space like a wiki where people always know what to search for?
I think there needs to be a linguist version of "what every programmer needs to know about (full?) text search"...
I'm not a linguist and I don't study languages, but I know enough to realize if a text search system is not designed for a particular language, it simply won't work. (As an example, to implement English search in a system for a hobby project, I had to import a US/UK spelling wordlist, and implement the Porter Stemming Algorithm. This is just for "one" language, and probably does not cover the other "English" dialects. Imagine doing a different workaround for every language in existence...)
RAG is actually a very language-agnostic way to work around those issues.
More LLM-generated text about LLMs. Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.
I'm Becoming AI-Blind: https://news.ycombinator.com/item?id=49386699