Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

181–190 of 197 posts

Re: RAG Is Simpler Than You Think

#181

Post by jrochkind1 »

j0selit0 wrote: post #137
cpdomina wrote: post #114

Earlier quoted context omitted.

the biggest giveway is actually not the writing style, but the content "using GPT-4o-mini for query rewriting" -> model from 2024, when RAG was trendy, and all the langchain, llama-index, etc, docs mentioned this specific model

author here. at most companies I've worked for recently (F500) RAG is still quite trendy. this was what frustrated me a bit and motivated to write this article - along with other experiences that definitely relate with some of the folks in the comments above

But you actually had an LLM write this article, no?

Re: RAG Is Simpler Than You Think

#182

Post by LukasWeber »

jrochkind1 wrote: post #37

More LLM-generated text about LLMs. Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.

"This is where the real trade-off lives – not cost, but speed."

That is where I stopped

Re: RAG Is Simpler Than You Think

#183

Post by woah »

mdp2021 wrote: post #170
andy99 wrote: post #140

Earlier quoted context omitted.

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.

That could work in a way, but it's very expensive as expressed and I do not know of prominent robust implementations. On the other hand, your post may contain a good idea: L=instruct_LLM("provide a list of synonyms and periphrases of terms T within context C", T, C); then iter(`grep l in L`). One NN query and a `grep` collection. But again, if one wanted to order the results, it is either through a dumb crierion or t…

You do not know of prominent robust implementations? This is how Claude Code, GPT Codex, etc have worked for a couple years. And they do tend to be impressively good at navigating large amounts of text.

Re: RAG Is Simpler Than You Think

#184

Post by wldcordeiro »

serf wrote: post #148
AshleyGrant wrote: post #101

Earlier quoted context omitted.

No. It isn't. With acronyms, there's often plenty of potential things it can stand for, and if the person doesn't know enough to know which one is the correct acronym, Googling it isn't going to help them. As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK." The NWS knows this and automatically links to their glossary for both acronyms as well as jargon…

>As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK." it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing. if you're reading this stuff, and you can't figure out what kind of RAG that the search engine mentioned is being talked about through context clues, or you aren't clever enough to feed context into the search…

blah blah blah justifications for gatekeeping.

Re: RAG Is Simpler Than You Think

#185

Post by locknitpicker »

dominotw wrote: post #134
locknitpicker wrote: post #112

Earlier quoted context omitted.

> It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours. It is, if people don't even stop to think if they need synonyms, periphrases, or mistaken neighbours. As the blog post points out, more often than not you don't, particularly if your primary usecase is to search for technical keywords or codenames.

> particularly if your primary usecase is to search for technical keywords or codenames. i dont believe ppl are building rag for this

> i dont believe ppl are building rag for this

What do you actually think people do when using LLMs to build AI coding agents?

Re: RAG Is Simpler Than You Think

#186

Post by eureka7 »

dominotw wrote: post #134
locknitpicker wrote: post #112

Earlier quoted context omitted.

> It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours. It is, if people don't even stop to think if they need synonyms, periphrases, or mistaken neighbours. As the blog post points out, more often than not you don't, particularly if your primary usecase is to search for technical keywords or codenames.

> particularly if your primary usecase is to search for technical keywords or codenames. i dont believe ppl are building rag for this

They are, I have people at work building RAG search engines for stuff that works just fine using full text search, or if you really need it, using a cheap model in codex/opencode.

You underestimate the ability of people to overengineer things.

Re: RAG Is Simpler Than You Think

#187

Post by IanCal »

jillesvangurp wrote: post #60

RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector search as magic pixie dust that makes search great without effort is not necessarily going to work that well. Also, it can add a lot of cost and complexity to the equation. And if not tuned properly, you don't necessarily get good results. The key thing with R…

> RAG is basically good old information retrieval with LLMs doing the querying.

No - rag is doing search before you call the llm to give it context from some corpus like your helpdesk articles.

Re: RAG Is Simpler Than You Think

#188

Post by bityard »

saltysalt wrote: post #111

If like me you run models locally, it's pretty easy to run your own RAG locally also using a Vector Database like Qdrant for persistence, and a middle-layer like Mem0 for realtime retrial and updates. I documented the set-up steps here: https://leadprompt.sh/a/739-Building-an-Infinite-Memory-Loca...

Thanks for the nice article.

If you're looking for feedback, I'd suggest adding a short demo at the end. It would be nice to see you send it a prompt that says, "hey, remember this" and then tell it to recall that memory. Or show what the memories look like on their way to the model. Are the memories added to the context on every turn or only once per conversation?

Re: RAG Is Simpler Than You Think

#189

Post by comandillos »

usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

I indexed thousands of documents into a SQLite database with an FTS5 index, plugged it into DeepSeek v4 Flash and got better much better results than any other commercial solutions my company has tried in the past.

The trick was just to let the LLM come up with its own SQL queries for searching... and the results are impressive.

Re: RAG Is Simpler Than You Think

#190

Post by Silasdev »

j0selit0 wrote: post #141
Silasdev wrote: post #116

Very little of this is RAG but rather just FTS with clever reformulation and re-ranking. RAG is about providing an grounded response, given the actual data in the corpus. Great article and content, nonetheless!!

author here - thank you!

I apologize for write "just" FTS. I know there is a lot of work involved and your article summed it up pretty damn well, in a way that makes it approachable for someone new to the topic.

I will keep a note of this article for next time I am asked about this topic.

Post reply on HN