Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Build a search engine, not a vector DB

blog.elicit.com

Post reply on HN
Print view View original

51–60 of 84 posts

Re: Build a search engine, not a vector DB

#51

Post by finikytou »

softwaredoug wrote: post #46

I feel like we're passing the peak of a vector db hype cycle, where its increasingly clear its one retrieval strategy next to full-text search strategies. I constantly talk to people trying to build RAG and they realize they need a full-text search solution, and a number of strategies, VERY dependent on the task you want your chat system to accomplish. It's important we get through the trough of disillusionment quick…

full text search is also overhyped. at the end you querying a KB just like in the 90s. the major difference is the scale of the model and the fact that he can make assumptions with a tone that would make you believe what is he saying is a fact

Re: Build a search engine, not a vector DB

#52

Post by bioxept »

It seems to me that the buzz-word "vector db" leads to people not fully understanding what it's actually about and how it even relates with LLMs. Vector databases or nearest neighbor algorithms (as they were called before) were already in use for lots of other tasks not related to language processing. If you look at them from that perspective, you will naturally think of vector dbs as just another way of doing plain old search. I hope we get some more advancements in hybrid search. Most of the times, search is the limiting factor when doing RAG.

Re: Build a search engine, not a vector DB

#53

Post by themanmaran »

softwaredoug wrote: post #46

I feel like we're passing the peak of a vector db hype cycle, where its increasingly clear its one retrieval strategy next to full-text search strategies. I constantly talk to people trying to build RAG and they realize they need a full-text search solution, and a number of strategies, VERY dependent on the task you want your chat system to accomplish. It's important we get through the trough of disillusionment quick…

I fell into this trap as well. Started pretty hyped about vector dbs as the "magical crtl+f". Realized I needed some keyword matching as well. And also some transforms to get the right format for vector search. And also multiple chunking strategies for more fidelity search.

A month in I realize I'm trying to reinvent a search engine. Kinda wonder if I should have just used something like elasticsearch instead.

Re: Build a search engine, not a vector DB

#54

Post by PaulHoule »

marginalia_nu wrote: post #29
bambax wrote: post #17

Many, many big companies don't see any value in search. They simply use the defaults, and when those defaults are abysmal (like in the case of Confluence for example), well... they just suffer through it in silence. I have so far mostly failed in trying to explain 1/ why search matters and 2/ that not all "search" functionality are equal and that building good search is an art form.

> I have so far mostly failed in trying to explain 1/ why search matters and 2/ that not all "search" functionality are equal and that building good search is an art form. Yeah, it takes an absurd amount of tuning to make search work well. Given how poorly the average search field works in almost anything, it's fair to say this crucial step isn't happening. I suspect a lot of organizations just don't have workflows t…

I went through a phase where I spoke to people who develop numerous enterprise search engines (e.g. OpenText) out of about 20 interviews I think I found one that did actual evaluation work on their search engine. The rest of them figured it was more important to have 300+ 'integrations' to various data sources and didn't think the relevance of the results was much of a selling point.

Re: Build a search engine, not a vector DB

#55

Post by james-revisoai »

bioxept wrote: post #52

It seems to me that the buzz-word "vector db" leads to people not fully understanding what it's actually about and how it even relates with LLMs. Vector databases or nearest neighbor algorithms (as they were called before) were already in use for lots of other tasks not related to language processing. If you look at them from that perspective, you will naturally think of vector dbs as just another way of doing plain…

Good points... In many ways, before LLMs, vectors were getting so exciting, Sentence Transformers and BERT embeddings felt so instrumental, so powerful... work by the txtai author (especially things like semantic walking) felt incredible and like the next evolution. It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go into product before ChatGPT made so much except search use cases obsolete..

Re: Build a search engine, not a vector DB

#57

Post by VivaLaPanda »

politelemon wrote: post #47

> you could have a language model construct a query that includes a date filter. But be careful because the output is not guaranteed. Which means you have to take care to provide the schema and what you're trying to do within the context window, and validate the output. There is a non-trivial overhead to this.

OAI function calling can solve this more or less

Re: Build a search engine, not a vector DB

#58

Post by VivaLaPanda »

avereveard wrote: post #2

Agree fully, vector search in embedding space is insufficient if you are working wirh a single document domain (i.e. They are all fish restaurant menu) and then the only thing that can save you is text search. Just make sure the underlying database supports synonyms lists and normalization in the languages you plan using. About the "bad news" section. You can do that today by just asking the llm using the ReAct patte…

I think a model could do some basic eval but there are too many hidden assumptions for it to do especially well.

Re: Build a search engine, not a vector DB

#59

Post by bodantogat »

m-i-l wrote: post #23
jankovicsandras wrote: post #7

I agree too. My impression is that almost all RAG tutorials _only_ talk about vector DBs, when these are not strictly required for Retrieval Augmented Generation. I'm guessing vector DBs are useful when you have massive amounts of documents on diverse topics. Some gotchas I experienced (but I might be using the wrong embedding/vector DB: spaCy/FAISS): - Short user questions might result a low signal query vector, e.…

> "Has somebody experience with Apache Lucene / Solr or Elasticsearch?" I've been working on a RAG with Solr, and quickly hit some of the issues you describe when dealing with real-world messy data and user input, e.g. using all-MiniLM-L6-v2 and cosine similarity, "Can you summarize Immanuel Kant's biography?" matched a chunk containing just the word "Biography" rather than one which started "Immanuel Kant, born in 1…

Exactly in the same place as you with Elastic Search (8.11). Went down the vector path to get better matches for adjectives, verbs and negations ( "room with no skylight" vs. "room with skylights" & "room with a large skylight"). Different dataset obviously, but I think I get slightly better results than your examples and it might be worth looking for a different sentence transformer (I tried a few and settled on roberta-base-nli-stsb-mean-tokens).

Re: Build a search engine, not a vector DB

#60

Post by deckar01 »

sroussey wrote: post #15
sroussey wrote: post #13

Earlier quoted context omitted.

Ask the LLM to summarize the question, then take an embedding of that. I think you can do the same with data you store… summarize it to same number of tokens, then get an embedding for that to save with the original text. Test! Different combinations of summarizing LLM and embedding generation LLM can get different results. But once you decide, you are locked in the summarizer as much as the embedding generator. Not…

BTW: I think of this like asking someone to put things into their own words, and then it’s easier for them to remember. Matching on your way of talking can be weird from the LLM’s point of view, so use their point of view!

It is two different language models. The embedding model tries to capture too many irrelevant aspects of the prompt that ends up putting it close to seemingly random documents. Inverting the question into the LLM’s blind guess and distilling it down to keywords causes the embedding to be very sparse and specific. A popular strategy has been to invert the documents into questions during initial embedding, but I think that is a performance hack that still suffers from sentence prompts being bad vector indexes.
Post reply on HN