Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

51–60 of 111 posts

Re: RAG Is Simpler Than You Think

#51

Post by robrorcroptrer »

usernametaken29 wrote: post #29
refactor_master wrote: post #14

Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls. Then, throw it all in BigQuery. Handles all the vector stuff natively. Sprinkle an agentic bot UI thing on top to make it appear…

> embed everything the first time This assumes your text is small. Try embedding pdf reports - though luck. It surely won’t fit into most embeddings. I can think of many more examples: books, news articles, medical reports, insurance claims etc. they’re all too big to “index it all at once”

What about splitting bigger content into chunks before embedding?

Re: RAG Is Simpler Than You Think

#53

Post by ninkendo »

Lorean1 wrote: post #36
vaylian wrote: post #32

Earlier quoted context omitted.

A hyperlink to Wikipedia would have solved that issue.

Maybe if a person can't even google RAG they are not the intended audience of that article.

When I hear stuff like this I always imagine going to a restaurant and asking the waitress for a menu and them replying “lol just google it”.

It’s not that I can’t or don’t know how, it’s rather that the expectation should be that a website should… link you to the information it believes to be relevant background. It’s why it’s called a “web”, linking is a core concept.

Re: RAG Is Simpler Than You Think

#54

Post by tux3 »

Zambyte wrote: post #50
Lorean1 wrote: post #36

Earlier quoted context omitted.

Maybe if a person can't even google RAG they are not the intended audience of that article.

Eh, a healthy web is a web. I enjoy my preferred search engine, but surfing the web is becoming a lost medium.

Hypermedia? In my hypertext markup language?

That is so not Web 5.0. Best I can offer is a support widget that pops up and keeps trying to talk to you until you interract with it.

Re: RAG Is Simpler Than You Think

#57

Post by 7734128 »

dsego wrote: post #22
apavlinovic wrote: post #7

The article sounds like AI slop with some predictable tells like short punctual sentences, bizarre jargon, and titles like "Recipe 4: On-The-Fly Embedding (The Fresh Data Play)" Can we not reward junk like this? Most of the sentences are incomprehensible and provide zero actual argumentation, it's just a list of "whats" with no "whys"

You are right, now I noticed "Real talk" and "Why this is underrated" and I can't unsee it.

They're absolutely right – and this is is why it's a load bearing observation that cuts to the heart of the issue.

Re: RAG Is Simpler Than You Think

#59

Post by Foobar8568 »

ewidar wrote: post #24
brabel wrote: post #11

Earlier quoted context omitted.

The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

not really, vectorising text/books is old school ML by this point. at least to me that seems the same as https://en.wikipedia.org/wiki/Word2vec for e.g.

Well... Everything new is old "A vector space model for automatic indexing" 1975 - https://dl.acm.org/doi/10.1145/361219.361220

Re: RAG Is Simpler Than You Think

#60

Post by jillesvangurp »

RAG is basically good old information retrieval with LLMs doing the querying. This can include vector search but it works without that as well. Treating vector search as magic pixie dust that makes search great without effort is not necessarily going to work that well. Also, it can add a lot of cost and complexity to the equation. And if not tuned properly, you don't necessarily get good results.

The key thing with RAG is to get the right information in the context with as few queries as possible. That requires good recall (ensuring that if it is there it can be found with a reasonable query) and precision (ensuring the best stuff is on top and minimizing false positives).

With search, and by extension RAG, the principle of shit in, shit out applies. Most of what search teams did before AI and RAG is still the best way to optimize the experience with RAG. And if you mess that up, search is not going to be working that well and no amount of AI can compensate for that or only at great cost in tokens and time. So, having an ETL pipeline to pre-process what you index, testing & benchmarking search quality, etc. are all helpful.

The good news is that you don't need that much skills with agentic coding to build something half decent for this. This code almost writes itself. And even a little bit of effort on extracting structure before indexing can make a big difference.

Post reply on HN