RAG is retrieval-augmented generation. I had never heard of this before.
Build a search engine, not a vector DB
61–70 of 84 posts
Re: Build a search engine, not a vector DB
#62Earlier quoted context omitted.
> I have so far mostly failed in trying to explain 1/ why search matters and 2/ that not all "search" functionality are equal and that building good search is an art form. Yeah, it takes an absurd amount of tuning to make search work well. Given how poorly the average search field works in almost anything, it's fair to say this crucial step isn't happening. I suspect a lot of organizations just don't have workflows t…
I went through a phase where I spoke to people who develop numerous enterprise search engines (e.g. OpenText) out of about 20 interviews I think I found one that did actual evaluation work on their search engine. The rest of them figured it was more important to have 300+ 'integrations' to various data sources and didn't think the relevance of the results was much of a selling point.
Quality is harder to sell to enterprise customers when compared to feature lists. You have to check the right boxes and entertain the right ears to sell.
Being more useful than the others isn't as easy to quantify.
Re: Build a search engine, not a vector DB
#63I feel like we're passing the peak of a vector db hype cycle, where its increasingly clear its one retrieval strategy next to full-text search strategies. I constantly talk to people trying to build RAG and they realize they need a full-text search solution, and a number of strategies, VERY dependent on the task you want your chat system to accomplish. It's important we get through the trough of disillusionment quick…
full text search is also overhyped. at the end you querying a KB just like in the 90s. the major difference is the scale of the model and the fact that he can make assumptions with a tone that would make you believe what is he saying is a fact
I disagree it’s “overhyped”. I feel like there’s a fairly correct understanding in the market of its uses and limitations. That hype cycle occurred decades ago
Re: Build a search engine, not a vector DB
#64I feel like we're passing the peak of a vector db hype cycle, where its increasingly clear its one retrieval strategy next to full-text search strategies. I constantly talk to people trying to build RAG and they realize they need a full-text search solution, and a number of strategies, VERY dependent on the task you want your chat system to accomplish. It's important we get through the trough of disillusionment quick…
I fell into this trap as well. Started pretty hyped about vector dbs as the "magical crtl+f". Realized I needed some keyword matching as well. And also some transforms to get the right format for vector search. And also multiple chunking strategies for more fidelity search. A month in I realize I'm trying to reinvent a search engine. Kinda wonder if I should have just used something like elasticsearch instead.
It's worth saying that ES can work as a vector store itself, so it's very easy to handle a couple different kinds of searches this way.
Re: Build a search engine, not a vector DB
#65Earlier quoted context omitted.
I went through a phase where I spoke to people who develop numerous enterprise search engines (e.g. OpenText) out of about 20 interviews I think I found one that did actual evaluation work on their search engine. The rest of them figured it was more important to have 300+ 'integrations' to various data sources and didn't think the relevance of the results was much of a selling point.
Quality is harder to sell to enterprise customers when compared to feature lists. You have to check the right boxes and entertain the right ears to sell. Being more useful than the others isn't as easy to quantify.
[deleted]
Re: Build a search engine, not a vector DB
#66It seems to me that the buzz-word "vector db" leads to people not fully understanding what it's actually about and how it even relates with LLMs. Vector databases or nearest neighbor algorithms (as they were called before) were already in use for lots of other tasks not related to language processing. If you look at them from that perspective, you will naturally think of vector dbs as just another way of doing plain…
Good points... In many ways, before LLMs, vectors were getting so exciting, Sentence Transformers and BERT embeddings felt so instrumental, so powerful... work by the txtai author (especially things like semantic walking) felt incredible and like the next evolution. It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go i…
>It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go into product before ChatGPT
Yes, it did. Companies that offer competitive search or recommendation feeds were all using these text models in production.
Re: Build a search engine, not a vector DB
#67Earlier quoted context omitted.
BTW: I think of this like asking someone to put things into their own words, and then it’s easier for them to remember. Matching on your way of talking can be weird from the LLM’s point of view, so use their point of view!
It is two different language models. The embedding model tries to capture too many irrelevant aspects of the prompt that ends up putting it close to seemingly random documents. Inverting the question into the LLM’s blind guess and distilling it down to keywords causes the embedding to be very sparse and specific. A popular strategy has been to invert the documents into questions during initial embedding, but I think…
You can use llama2 to do embedding and summaries and chat.
Turning the docs into questions is something I will test on stuff (just learning and getting a feel).
I am intrigued... what makes a good vector index??
Re: Build a search engine, not a vector DB
#68Earlier quoted context omitted.
Good points... In many ways, before LLMs, vectors were getting so exciting, Sentence Transformers and BERT embeddings felt so instrumental, so powerful... work by the txtai author (especially things like semantic walking) felt incredible and like the next evolution. It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go i…
>It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go into product before ChatGPT Yes, it did. Companies that offer competitive search or recommendation feeds were all using these text models in production.
I was running one of them, and entering kaggle competitions throughout 2021 and 2022 using them. Many efforts and uses of Sentence-transformers (and new PhD projects) were thrown in the trash with Instruct GPT models and ChatGPT. I mean it's like developing a much better bicycle (lets say an ebike) but then cars come out. It was like that.
The future looked incredibly creative with cross-encoders, things like semantic paths, using the latent space to classify - everything was exciting. A all-in-one LLM that eclipsed embeddings on all but speed for these things was a bit of a kill joy.
Companies that changed existing indexing to use sentence transformers aren't exactly innovating; that process happened once or twice a decade for the last few decades. This was parents point I believe, in a way. And tbh, the improvement in results has never been noticeable to me; exact match is actually 90% of the solution to retrieval(maybe not search) already - we just take it for granted because we are so used to it.
I fully believe in a world without GPT-3, HN demos would be full of sentence transformer and other cool technology being used for demos and in creative ways, compared to how rarely you see them.
Re: Build a search engine, not a vector DB
#69Earlier quoted context omitted.
>It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go into product before ChatGPT Yes, it did. Companies that offer competitive search or recommendation feeds were all using these text models in production.
I was running one of them, and entering kaggle competitions throughout 2021 and 2022 using them. Many efforts and uses of Sentence-transformers (and new PhD projects) were thrown in the trash with Instruct GPT models and ChatGPT. I mean it's like developing a much better bicycle (lets say an ebike) but then cars come out. It was like that. The future looked incredibly creative with cross-encoders, things like semanti…
>Many efforts and uses of Sentence-transformers (and new PhD projects) were thrown in the trash with Instruct GPT models and ChatGPT.
There still exists a need for fast and cheap models where LLMs do not make sense.
Re: Build a search engine, not a vector DB
#70It seems to me that the buzz-word "vector db" leads to people not fully understanding what it's actually about and how it even relates with LLMs. Vector databases or nearest neighbor algorithms (as they were called before) were already in use for lots of other tasks not related to language processing. If you look at them from that perspective, you will naturally think of vector dbs as just another way of doing plain…
Good points... In many ways, before LLMs, vectors were getting so exciting, Sentence Transformers and BERT embeddings felt so instrumental, so powerful... work by the txtai author (especially things like semantic walking) felt incredible and like the next evolution. It's a shame in a way that all the creative and brilliant uses of text embeddings from similarity embeddings didn't really have any time to shine or go i…
Btw - I published a paper at EMNLP with the txtai author (David) about using semantic graphs for automatic creation of debate cases!
https://aclanthology.org/2023.newsum-1.10/
Happy to see that David's excellent work is getting the love that it deserves!