Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

11–20 of 144 posts

Re: RAG Is Simpler Than You Think

#11

Post by brabel »

nilirl wrote: post #4

Maybe I'm old but where exactly are the "dragons"? How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend? If so, I'd like to see more design patterns around existing search problems: - Correcting or backtracking based on feedback. - Measuring relevance. - Comparison with task-based pre-written que…

The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

Re: RAG Is Simpler Than You Think

#12

Post by dizhn »

petesergeant wrote: post #8
7734128 wrote: post #2

There have been many blogs like this over the last years. Yes, embeddings are computationally heavy, but they are not at all complicated and they provide a lot of benefit. 90% of "document" based RAG projects should view semantic search with embeddings as their primary method. It's very powerful and so easy to implement that you could try it out and discover whether performance would be an issue rather than trying to…

Embeddings are reasonably simple, but it’s a journey to get there, and I am very proud of the dog-heavy explainer I wrote on them: https://sgnt.ai/p/embeddings-explainer/

This is very good. Thanks.

Re: RAG Is Simpler Than You Think

#14

Post by refactor_master »

Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls.

Then, throw it all in BigQuery. Handles all the vector stuff natively.

Sprinkle an agentic bot UI thing on top to make it appear all-knowing and magical.

I assume other vendors than Google have a similar batteries-included approach you can just plug in.

Re: RAG Is Simpler Than You Think

#15

Post by bob1029 »

Agentic query rewrite on top of good old fashioned Lucene is the end game. This is effectively providing a lot of the same magic you get with the semantic approach. Allowing the agent to query the document store iteratively is where the capabilities become unbounded.

Embeddings and semantic search add non determinism on top of non determinism. This seems fundamentally cursed. Lexical is much easier to control, iterate and debug. The tools are incredibly mature. Your users will probably prefer it as well.

Re: RAG Is Simpler Than You Think

#16

Post by jankovicsandras »

If someone has a Postgres db and want very simple RAG:

https://github.com/jankovicsandras/plpgsql_bm25 BM25 search implemented in PL/pgSQL ( Unlicense / Public domain )

The repo includes also plpgsql_bm25rrf.sql : PL/pgSQL function for hybrid search ( plpgsql_bm25 + pgvector ) with Reciprocal Rank Fusion; and Jupyter notebook examples.

Re: RAG Is Simpler Than You Think

#18

Post by MacketSWE »

nilirl wrote: post #4

Maybe I'm old but where exactly are the "dragons"? How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend? If so, I'd like to see more design patterns around existing search problems: - Correcting or backtracking based on feedback. - Measuring relevance. - Comparison with task-based pre-written que…

[flagged]

Re: RAG Is Simpler Than You Think

#19

Post by triangle »

brabel wrote: post #11
nilirl wrote: post #4

Maybe I'm old but where exactly are the "dragons"? How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend? If so, I'd like to see more design patterns around existing search problems: - Correcting or backtracking based on feedback. - Measuring relevance. - Comparison with task-based pre-written que…

The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

Vector embeddings predate LLMs. They have been used as far back as the early 2000s. They are a general machine learning technique, rather than LLM specific

Re: RAG Is Simpler Than You Think

#20

Post by nilirl »

brabel wrote: post #11
nilirl wrote: post #4

Maybe I'm old but where exactly are the "dragons"? How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend? If so, I'd like to see more design patterns around existing search problems: - Correcting or backtracking based on feedback. - Measuring relevance. - Comparison with task-based pre-written que…

The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

Sure and that's a new technique for indexing and querying.

Where's the new design tension? Indexes always had to be monitored for freshness and queries have always needed cleaning or parsing.

Post reply on HN