Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

131–140 of 189 posts

Re: RAG Is Simpler Than You Think

#131

Post by lopsotronic »

locknitpicker wrote: post #112
mdp2021 wrote: post #103

Earlier quoted context omitted.

> people vastly underestimate full text search It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

> It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours. It is, if people don't even stop to think if they need synonyms, periphrases, or mistaken neighbours. As the blog post points out, more often than not you don't, particularly if your primary usecase is to search for technical keywords or codenames.

Precisely this. The people in charge of technical direction don't understand the fundamentals of the technology. So you get the idea that LLMs can help make sense of parts data. Which . . . no, no it really can't, not without ALSO plugging in basically every other hunk of natural language you might have laying around. Unless you think PLG HT HFI is just a natural synonym of HOT PLUG INJECTOR, in which case you're just quantitatively wrong.

Vectors and LLMs are great, but there's no magic pill here. If your parts data and config management[1] is all crazy, that's an institutional problem. Buying a crapton of tokens isn't fixing it, unless you're using it to help build an actual formal solution based on good fundamentals.

[1] Such as it is.

Re: RAG Is Simpler Than You Think

#132

Post by timedude »

cpdomina wrote: post #114
jrochkind1 wrote: post #37

More LLM-generated text about LLMs. Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.

the biggest giveway is actually not the writing style, but the content "using GPT-4o-mini for query rewriting" -> model from 2024, when RAG was trendy, and all the langchain, llama-index, etc, docs mentioned this specific model

GPT-4o-mini is still very relevant. It is a very capable model and very very fast and dirt cheap. Excellent for this type of stuff

Re: RAG Is Simpler Than You Think

#133

Post by Angostura »

dotancohen wrote: post #25
Angostura wrote: post #6

I have a particular antipathy for articles too lazy to spell out acronyms on first use. So: https://en.wikipedia.org/wiki/Retrieval-augmented_generation

The audience for this piece is already very familiar with RAG. I don't want articles discussing e.g. OLED screens telling me what the acronym is - that would be a sign that the article is far below the level that I need.

I found the piece interesting, once I worked out what it was about. I strongly disagree that taking time to spell out acronyms should be taken as a signal that an article is low level.

Re: RAG Is Simpler Than You Think

#134

Post by dominotw »

locknitpicker wrote: post #112
mdp2021 wrote: post #103

Earlier quoted context omitted.

> people vastly underestimate full text search It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

> It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours. It is, if people don't even stop to think if they need synonyms, periphrases, or mistaken neighbours. As the blog post points out, more often than not you don't, particularly if your primary usecase is to search for technical keywords or codenames.

> particularly if your primary usecase is to search for technical keywords or codenames.

i dont believe ppl are building rag for this

Re: RAG Is Simpler Than You Think

#135

Post by harlanji »

gf000 wrote: post #117
freakynit wrote: post #81

Earlier quoted context omitted.

How are you gonna handle the relations that span across individual chunks... if a later chunk refers something from 2 chunks before using `it`, rather than proper name, how will you handle that? Because at query time, that later chunk would not match.

Absolutely a novice in this topic, but I would imagine that by simply having sufficiently big chunks it's simply not a problem? You surely have enough information in like a couple of paragraphs to denote in vector space roughly what it is about. So that both chunks would get found by a vector search, and then whatever is the logic it may put the whole original text of those chunks into context, but in any case enough…

Chunks can only be as large as the embedding model’s token limit, about 512-1024 tokens usually. Anything longer gets truncated.

Natural language processing could expanded references, but it starts to get tricky. Do you use Graph RAG, embed another version of the chunk that is distinct from the full text version, etc.. Another layer of processing and data to keep in sync if the source dan be updated.

Re: RAG Is Simpler Than You Think

#137

Post by j0selit0 »

cpdomina wrote: post #114
jrochkind1 wrote: post #37

More LLM-generated text about LLMs. Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.

the biggest giveway is actually not the writing style, but the content "using GPT-4o-mini for query rewriting" -> model from 2024, when RAG was trendy, and all the langchain, llama-index, etc, docs mentioned this specific model

author here. at most companies I've worked for recently (F500) RAG is still quite trendy. this was what frustrated me a bit and motivated to write this article - along with other experiences that definitely relate with some of the folks in the comments above

Re: RAG Is Simpler Than You Think

#138

Post by j0selit0 »

refactor_master wrote: post #14

Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls. Then, throw it all in BigQuery. Handles all the vector stuff natively. Sprinkle an agentic bot UI thing on top to make it appear…

I'm sorry is this ironic or not? doesn't sounds simple at all

Re: RAG Is Simpler Than You Think

#139

Post by j0selit0 »

yipinwong wrote: post #128

Only those who mastered the craft makes their work look simple. The AI that wrote this might be the master not the writer, as this looks written by AIs. I will use the author's agents, not read his articles or use him for the job.

author here - thanks, I'm honored you would use my agents :)

Re: RAG Is Simpler Than You Think

#140

Post by andy99 »

mdp2021 wrote: post #103
usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

> people vastly underestimate full text search It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.
Post reply on HN