Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

161–170 of 184 posts

Re: RAG Is Simpler Than You Think

#161

Post by AshleyGrant »

serf wrote: post #148
AshleyGrant wrote: post #101

Earlier quoted context omitted.

No. It isn't. With acronyms, there's often plenty of potential things it can stand for, and if the person doesn't know enough to know which one is the correct acronym, Googling it isn't going to help them. As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK." The NWS knows this and automatically links to their glossary for both acronyms as well as jargon…

>As OP said, simply providing a link to a Wikipedia article, or a glossary, helps widen the audience beyond "IFYKYK." it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing. if you're reading this stuff, and you can't figure out what kind of RAG that the search engine mentioned is being talked about through context clues, or you aren't clever enough to feed context into the search…

> it also serves as a minimum barrier to entry for the masses, which isn't always a bad thing.

No. Lowering the barrier of entry to those who are trying, in earnest, to learn about a new topic, to broaden their base of knowledge is NEVER a bad thing. None of us were born with the knowledge to read this (or similar) article. Trying to kick the ladder down after you have climbed it is terrible behavior and absolutely must be discouraged and stopped at all costs.

> much like this stupid divergence

The only thing stupid in this conversation is the insistence that folks who might know less than the author of the linked article or the poster do not deserve to have access to the information.

Re: RAG Is Simpler Than You Think

#162

Post by IronyMan1 »

usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

I believe the second Suggestion solves 95% of my problems. I want a system where i can describe my search and the system generated 5-15 keywords for a query

Re: RAG Is Simpler Than You Think

#163

Post by triceratops »

dotancohen wrote: post #25
Angostura wrote: post #6

I have a particular antipathy for articles too lazy to spell out acronyms on first use. So: https://en.wikipedia.org/wiki/Retrieval-augmented_generation

The audience for this piece is already very familiar with RAG. I don't want articles discussing e.g. OLED screens telling me what the acronym is - that would be a sign that the article is far below the level that I need.

There's a middle-ground where you write out ambiguous acronyms ("rag" is an English word) and not unambiguous ones ("oled" only has one commonly-used meaning).

Re: RAG Is Simpler Than You Think

#164

Post by jillesvangurp »

dmix wrote: post #142
jillesvangurp wrote: post #60

RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector search as magic pixie dust that makes search great without effort is not necessarily going to work that well. Also, it can add a lot of cost and complexity to the equation. And if not tuned properly, you don't necessarily get good results. The key thing with R…

> With search, and by extension RAG, the principle of shit in, shit out applies Similar to SEO on marketing pages, we started rewriting product docs around the idea that it will be consumed by a RAG. Mostly by putting a lot of focus on well structured headlines, thinking more carefully about technical terminology vs common human-language questions, occasionally using variations of keywords in the text, etc. This appl…

It's a start. Where it gets tricky is companies with years/decades of highly unstructured data, duplicated documents, obsolete or draft versions of those documents, etc. And where it gets more tricky if the data is spread all over the place in weird tools, databases, spreadsheets, etc. that has some structure but is maybe a bit inconsistent, incomplete, or not that well documented.

If you flatten all that into plain text and then create embeddings, you are effectively throwing out the baby with the bathwater. But on the other hand if you put some effort into normalizing and extracting some structured meta data, you gain a flexibility to do more sophisticated querying that get you more precise results.

You can of course try to fix things at the source, which is a valid thing but usually not that practical when you have a lot of data to worry about.

Re: RAG Is Simpler Than You Think

#165

Post by yipinwong »

j0selit0 wrote: post #139
yipinwong wrote: post #128

Only those who mastered the craft makes their work look simple. The AI that wrote this might be the master not the writer, as this looks written by AIs. I will use the author's agents, not read his articles or use him for the job.

author here - thanks, I'm honored you would use my agents :)

I'd still pay for your brain (expertise, experience for built them), which is separate from your agent :)

Re: RAG Is Simpler Than You Think

#166

Post by inigyou »

respectattentio wrote: post #157
inigyou wrote: post #99

Earlier quoted context omitted.

How would you use a big LLM for retrieval?

As simple as a prompting it with structural output or restrictions for your criteria. With agents, the prompting could be dynamic for maximum accuracy for every retrieval. This absolutely would beat the best of the best embedding-based RAG models. Nobody uses this now mainly due to speed. An llm retrieval would be 10x or more slower than embedding. You can try that now Take some failing cases or bad retrieval from yo…

So uh ... Where's the retrieval part? You know RAG is used to implement that, right? You're basically saying "we don't need an ALU, we can just use the Windows calculator"

Re: RAG Is Simpler Than You Think

#167

Post by 0x457 »

spunker540 wrote: post #160
klm127 wrote: post #124

RAG stands for Retrieval Augmented Generation. The purpose is to search a corpus of text by meaning rather than exact match. I had to look it up.

That sounds more like semantic search and vector db. RAG is simply fetching external data (retrieval) and adding it to LLM context (augmenting) prior to generating a final response. Any time LLMs do a grep or a web search to answer the query, it’s RAG. Many people use vector db for their own RAG implementation bc of the semantic search benefits.

Because people writing about RAG never explained what RAG is and exclusively wrote about embeddings and vector dbs, for most people RAG became "embeddings + vector db".

People don't understand that any sort of retrieval before generation is RAG.

Re: RAG Is Simpler Than You Think

#168

Post by seamossfet »

I notice a lot of these AI written articles share this pattern where they'll present idea 1, then idea 2, and finally idea 3 which is some amalgamation of idea 1 and 2. Claude especially will present hybrid options and compromises to avoid having to make a choice then framing the hybrid option as the "best of both worlds" when they're borderline nonsensical.

"on the fly embedding" and "Sparse + dense reranking" don't really make sense how they're presented and smell like they came from a long claude-driven conversation after multiple cycles of these hybrid compromises across many turns.

Re: RAG Is Simpler Than You Think

#169

Post by ChipopLeMoral »

seamossfet wrote: post #168

I notice a lot of these AI written articles share this pattern where they'll present idea 1, then idea 2, and finally idea 3 which is some amalgamation of idea 1 and 2. Claude especially will present hybrid options and compromises to avoid having to make a choice then framing the hybrid option as the "best of both worlds" when they're borderline nonsensical. "on the fly embedding" and "Sparse + dense reranking" don't…

This has Claude written all over it.

"Recipe 4: On-The-Fly Embedding (The Fresh Data Play)

The insight If your data changes frequently, why pay to re-embed everything?"

This reads like every Claude generated presentation I've seen.

Re: RAG Is Simpler Than You Think

#170

Post by mdp2021 »

andy99 wrote: post #140
mdp2021 wrote: post #103

Earlier quoted context omitted.

> people vastly underestimate full text search It is not psychological, it is fully justified: substring search cannot find synonyms, periphrases and mistaken neighbours.

Maybe I’m interpreting this differently but to me modern LLM+full text search means “agentic” - LLM gets to pick the search terms and iterate on them. The underlying LLM does know synonyms etc, better and more flexibly than an embedding model, and gets explainable feedback from failed searches.

That could work in a way, but it's very expensive as expressed and I do not know of prominent robust implementations.

On the other hand, your post may contain a good idea: L=instruct_LLM("provide a list of synonyms and periphrases of terms T within context C", T, C); then iter(`grep l in L`). One NN query and a `grep` collection. But again, if one wanted to order the results, it is either through a dumb crierion or through another LLM query - but this could make it extremely costly (requiring either a huge context or a quadratic number of ordering queries).

And, the above `grep` based procedure would remain keyword based and not semantic based, which means that the user must know that it will not be based on comprehension but on the possible results that keyword matching can yield.

Post reply on HN