Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

71–80 of 126 posts

Re: RAG Is Simpler Than You Think

#71

Post by wongarsu »

lacedeconstruct wrote: post #30
usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

I thought text search was always the first thing you try, then fuzzy search, then you go for RAG

It's not like a simple embedding search takes that much longer to implement. Especially on short descriptions where you don't have to deal with chunking. And if you let an LLM write the code it's even less of a difference. Combine that with embedding search promising to solve all your search problems, and I understand why people often skip over full text search and go straight to embeddings

Re: RAG Is Simpler Than You Think

#72

Post by jrochkind1 »

pixelbro wrote: post #45
jrochkind1 wrote: post #39

Earlier quoted context omitted.

An LLM wrote this comment, no? I'm curious your motivation for having an LLM write such a short comment instead of writing it yourself?

I've not seen such a clipped cadence out of an LLM. I would not automatically suspect the GP. Maybe there's better ways to spend your time?

Maybe people are just learning to write in that style LLMs learned to write from statistical people? "is where the capabilities become unbounded" is a weird thing to say and not really true. "is the end game", "add non determinism on top of non determinism", there are a lot of AI-isms in this short comment. But it's possible people are just learning to write this way now, I am curious if that's so too!

As far as uses of time, you are engaging in this dialog too, if you find it not a good way to spend time I recommend ceasing!

Re: RAG Is Simpler Than You Think

#73

Post by jameshart »

usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

I think people also overestimate the need for full text search when the one doing the querying is an LLM. If your underlying data is structured records, like a customer database, while humans might not have time or skills to figure out that when they want to search by phone number they need to do a join from the contacts table to the users table and normalize the phone number to look up first, making it best to just surface phone numbers as part of the data that is full/text-indexed… an agent is quite happy to handcraft the right SQL to find records that match on a specific field, given the right SKILLS.md and schema information. Turning fuzzy searches into exact DB lookups is a great way LLMs can augment users.

(Obviously this doesn’t apply to searching actual rich document data - for that, go all in on text search, embedding, etc)

Re: RAG Is Simpler Than You Think

#75

Post by cpursley »

orisho wrote: post #23
cpursley wrote: post #21

Earlier quoted context omitted.

Yep, lock into some vendor from day 1. Great idea!

Vendor lock in is 2025. Porting became trivial with LLMs advancing like they have.

What I'm saying is pick transportable tech from day 1 so you can easily move if they shut down, hike prices, decide they don't like you, etc.

Re: RAG Is Simpler Than You Think

#76

Post by owen-hill »

imtringued wrote: post #26
simianwords wrote: post #17

OT but its interesting that none of the harnesses today use embeddings but just simple grep. I would not have predicted this

Ok? I'm not seeing how that is interesting, you're exclusively focusing on coding which requires precise substring locations. Google is basically almost entirely driven by embedding models now.

[flagged]

Re: RAG Is Simpler Than You Think

#77

Post by clevergadget »

usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

I don't know what level of quality is required for this site but RAG is trash its just trash. its magic beans.

Re: RAG Is Simpler Than You Think

#78

Post by ozim »

triangle wrote: post #19
brabel wrote: post #11

Earlier quoted context omitted.

The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

Vector embeddings predate LLMs. They have been used as far back as the early 2000s. They are a general machine learning technique, rather than LLM specific

Unfortunately LLMs made vector search more popular so it seems like something LLM specific.

What makes it worse, a lot of people in the thread equate vector search with RAG, whereas RAG is the name for anything that model can query so a user doesn't have to copy/paste feed it to the model manually like access to text files is RAG.

Re: RAG Is Simpler Than You Think

#79

Post by a1o »

ozim wrote: post #66
lacedeconstruct wrote: post #30

Earlier quoted context omitted.

I thought text search was always the first thing you try, then fuzzy search, then you go for RAG

I think Bitwarden implemented some vector search in their password search feature ... totally annoying it gives me back all kinds of stuff that I don't care. I want fuzzy search like 95% of time and then I might consider having additional list of things that can be suggested by vector search.

A good UI could do these and also exact match, give some point system to the results, then order them and perhaps use a bold highlight to reflect what parts of the input query reflected in each result.
Post reply on HN