Build a search engine, not a vector DB • Hacker News Board

Re: Build a search engine, not a vector DB

#81

Post by cyanydeez » Thu, Dec 21, 2023, 10:23 PM UTC

bodantogat wrote: post #59

m-i-l wrote: post #23

Earlier quoted context omitted.

> "Has somebody experience with Apache Lucene / Solr or Elasticsearch?" I've been working on a RAG with Solr, and quickly hit some of the issues you describe when dealing with real-world messy data and user input, e.g. using all-MiniLM-L6-v2 and cosine similarity, "Can you summarize Immanuel Kant's biography?" matched a chunk containing just the word "Biography" rather than one which started "Immanuel Kant, born in 1…

Exactly in the same place as you with Elastic Search (8.11). Went down the vector path to get better matches for adjectives, verbs and negations ( "room with no skylight" vs. "room with skylights" & "room with a large skylight"). Different dataset obviously, but I think I get slightly better results than your examples and it might be worth looking for a different sentence transformer (I tried a few and settled on rob…

was reading through open llama, looks like way to get pertinent results is via different ranking algorithm and score based on convergence. then shove that back into the LLM

Re: Build a search engine, not a vector DB

#82

Post by sroussey » Fri, Dec 22, 2023, 5:04 PM UTC

deckar01 wrote: post #75

sroussey wrote: post #67

Earlier quoted context omitted.

You can use llama2 to do embedding and summaries and chat. Turning the docs into questions is something I will test on stuff (just learning and getting a feel). I am intrigued... what makes a good vector index??

My heuristic is how much noise is in the closest vectors. Even if the top k matches seem good, if the following noise has practically identical distance scores, it is going to fail a lot in practice. Ideally you could calculate some constant threshold so that everything closer is relevant and everything further is irrelevant.

Apologies for being naive, but how do you calculate noise?

Re: Build a search engine, not a vector DB

#83

Post by bayesian_limit » Thu, Dec 28, 2023, 6:04 PM UTC

sroussey wrote: post #13

kristiandupont wrote: post #11

Earlier quoted context omitted.

"Search the embedding"? Could you elaborate on this, it sounds interesting!

Ask the LLM to summarize the question, then take an embedding of that. I think you can do the same with data you store… summarize it to same number of tokens, then get an embedding for that to save with the original text. Test! Different combinations of summarizing LLM and embedding generation LLM can get different results. But once you decide, you are locked in the summarizer as much as the embedding generator. Not…

I could not help but notice the Contriever curve is so much higher on y-axis Recall than the other methods (figure 11 in https://arxiv.org/pdf/2307.03172.pdf).

Has anyone come across more recent experiments, results, or papers related to this? I'm acquainted with the: - Contriever 2021 paper https://aclanthology.org/2021.eacl-main.74.pdf - Hyde 2022 https://arxiv.org/pdf/2212.10496.pdf

My suspicion is some pre-logic such as is the user's question dense enough then use Hyde with chat history. If anyone has more recent experience with Contrievers, would love to learn more about it!

Feel free to contact me directly on LinkedIn. https://www.linkedin.com/in/christybergman/

Re: Build a search engine, not a vector DB

#84

Post by codingjaguar » Wed, Jan 03, 2024, 5:15 AM UTC

codingjaguar wrote: post #79

Partially agree. Vector DBs are critical components in retrieval systems. What most applications need are retrieval systems, rather than building blocks of retrieval systems. That doesn't mean the building blocks are not important. As someone working on vector DB, I find many users struggling in building their own retrieval systems with building blocks such as embedding service (openai,cohere), logic orchestration fr…

Here is an article that systematically discusses how vector retrieval and BM25 affects the search quality, in another word, what kind of systems are the past, now and future:

https://thenewstack.io/the-transformative-fusion-of-probabil...