Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

RAG Is Simpler Than You Think

lighthousenewsletter.com

Post reply on HN
Print view View original

61–70 of 126 posts

Re: RAG Is Simpler Than You Think

#61

Post by vintermann »

ewidar wrote: post #24
brabel wrote: post #11

Earlier quoted context omitted.

The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.

not really, vectorising text/books is old school ML by this point. at least to me that seems the same as https://en.wikipedia.org/wiki/Word2vec for e.g.

Sure, the idea of making a vector embedding for words, sentences, documents etc. is old, but the meat is in how you construct this embedding. I think embeddings have gotten quite a bit better since word2vec.

Re: RAG Is Simpler Than You Think

#63

Post by jameshart »

kaon_2 wrote: post #48
jon-wood wrote: post #35

Earlier quoted context omitted.

Instinctively this feels like a two phase problem - start with some machine translation into a single spoken language and index that, then when people are querying do the same thing. When returning search results show them in the original language.

Yes we've tried. It works. But jargon is hard. RAG with embeddings works all the same. The LLM doesn't mind receiving sources in Italian, french and German, and then outputting the answer in Japanese while providing the verbatim German jargon term in brackets

Embedding search is effectively machine translation into a single common ‘language’ - embedding space - and then searching that; cleaner and less lossy than translating everything into English for searching, but harder to debug when it goes wrong.

Re: RAG Is Simpler Than You Think

#65

Post by Otterly99 »

Althought I agree with the first point of the author that FTS is underrated in this new RAG-first framework, the whole article really hides all the problems with RAG-pipeline and kind of hand wave everything.

If you are building a RAG pipeline for your company and are struggling like me, I would recommend this author that has whole series on entreprise documents (start with the one from May 22nd): https://towardsdatascience.com/author/angela.shi/page/4/

Note: I am not the author, just got her article in my newsletter and found it useful.

Re: RAG Is Simpler Than You Think

#66

Post by ozim »

lacedeconstruct wrote: post #30
usernametaken29 wrote: post #27

I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…

I thought text search was always the first thing you try, then fuzzy search, then you go for RAG

I think Bitwarden implemented some vector search in their password search feature ... totally annoying it gives me back all kinds of stuff that I don't care.

I want fuzzy search like 95% of time and then I might consider having additional list of things that can be suggested by vector search.

Re: RAG Is Simpler Than You Think

#67

Post by autogn0me »

gabosarmiento wrote: post #46

I would like to see how each recipe performs against its corresponding evals. Some sort of ranking would be useful. Everyone keeps posting articles about how to implement RAG, but I also wonder why there isn’t some sort of skill to help people create a simple retrieval plan, starting with the retrieval methods and connecting them with evals. This could show whether they actually improve the result and make retrieval…

It seems not many RAG compare themselves across the same benchmarks. https://ggozad.github.io/haiku.rag/ Does an ok job. The part I don’t see being discuss is the whole RL agents writing code to perform RAG queries. It’s one thing haiku-rag does that’s interesting and would like to know what other RAG have that agentic querying with benchmarks

Re: RAG Is Simpler Than You Think

#68

Post by Planktonne »

jrochkind1 wrote: post #37

More LLM-generated text about LLMs. Is anyone else actually finding it harder and harder to read LLM generated text? I find it quite tiring, my brain just does not want to get through it.

Your brain is incredibly adept at pattern recognition; it doesn't focus on LLM-generated text for the same reason it doesn't stare at wallpaper.

We've all learnt that it's not really communication, and so can be dispensed with.

Post reply on HN