Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introduction to vector similarity search (2022)

zilliz.com

Post reply on HN
Print view View original

21–30 of 57 posts

Re: Introduction to vector similarity search (2022)

#21

Post by drewcperkins »

jeffchuber wrote: post #6
behnamoh wrote: post #2

Can anyone please suggest a good stack for the following: - calculating text embeddings using open-source/local methods (not OpenAI) - storing them in a vector database. I'm confused by the myriad of options like Chromadb, Pinecone, etc. - running vector similarity search using open-source/local methods. Also, how granular should the text chunks be? Too short and we'll end up with a huge database, too long and we'll…

Really just chroma + openai is all you need. Chroma makes this easy https://docs.trychroma.com/embeddings . All you have to do is chunk... I'd start at a paragraph and experiment.

Chroma is just a wrapper for Clickhouse. I’d go with something like Weaviate or Qdrant that is a lot more mature as far as vector databases.

Re: Introduction to vector similarity search (2022)

#22

Post by cma »

Xenoamorphous wrote: post #8

I’m confused by the choice of word2vec for the embeddings, for “simplicity”. Using a transformer based model like Universal Sentence Encoder or SBERT is just as easy and the results will be considerably better.

Good enough for Google:

https://www.google.com/search?q=book+of+megadrive

In the TPUv4 paper they say they are doing word based embeddings for search, maybe explaining why Google search has gotten so bad.

Re: Introduction to vector similarity search (2022)

#23

Post by dunefox »

fzliu wrote: post #12
Xenoamorphous wrote: post #7

Earlier quoted context omitted.

This is what we use: BERT sentence transformers to generate the embeddings (we used Universal Sentence Encoder before that and it was good too), and ElasticSearch for storage, which has a dense vector data type. It also has a cosineSimilarity function to run searches.

I would not use Elastic for vector search due to its architectural limitations and poor performance when conducting vector search. https://zilliz.com/benchmark

What about Solr?

Re: Introduction to vector similarity search (2022)

#24

Post by weinzierl »

How would I decide on which granularity level I create my vectors? For example, if I create word level embeddings with word2vec and store one vector per word I probably will only get good results for keyword based queries, right?

If I create more fancy vectors, for example on the sentence level, it is not really clear to me how that is supposed to work.

Why is the vector embedding of a query, which typically has the sentence structure of a question, near the vector embedding of a propositional sentence that answers that query? Sure, it will probably not be completely off, just for the fact that query and answer will contain similar words, but how can that be better than just a fuzzy word search?

And finally: Should I decide on on particular level (like sentences) and store that or should I store word2vec and sentence and paragraph vectors in the same collection?

Re: Introduction to vector similarity search (2022)

#25

Post by fzliu »

weinzierl wrote: post #24

How would I decide on which granularity level I create my vectors? For example, if I create word level embeddings with word2vec and store one vector per word I probably will only get good results for keyword based queries, right? If I create more fancy vectors, for example on the sentence level, it is not really clear to me how that is supposed to work. Why is the vector embedding of a query, which typically has the…

I would not use word2vec in any application today - I used it in the blog post because it's a well-known model and because the embeddings it generates are static, i.e. one token always maps to the same embedding regardless of context.

If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/

Re: Introduction to vector similarity search (2022)

#26

Post by weinzierl »

fzliu wrote: post #25
weinzierl wrote: post #24

How would I decide on which granularity level I create my vectors? For example, if I create word level embeddings with word2vec and store one vector per word I probably will only get good results for keyword based queries, right? If I create more fancy vectors, for example on the sentence level, it is not really clear to me how that is supposed to work. Why is the vector embedding of a query, which typically has the…

I would not use word2vec in any application today - I used it in the blog post because it's a well-known model and because the embeddings it generates are static, i.e. one token always maps to the same embedding regardless of context. If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/

I understood that it was just an example and it is a good choice for an introductory text. My question is more into advice how to go on from that. For example, is SBERT a good choice if most of my queries are in fact multi-sentence paragraphs?

I guess what still doesn't add up in my mental model is how people seem to assume that the query embedding vector must somehow automatically be near good answer embedding vectors. I would have assumed that some conditions have to be met to make this true and I would be interested into advice in that regard, or alternatively an explanation why I'm wrong with my assumption.

Re: Introduction to vector similarity search (2022)

#27

Post by pizza »

weinzierl wrote: post #26
fzliu wrote: post #25

Earlier quoted context omitted.

I would not use word2vec in any application today - I used it in the blog post because it's a well-known model and because the embeddings it generates are static, i.e. one token always maps to the same embedding regardless of context. If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/

I understood that it was just an example and it is a good choice for an introductory text. My question is more into advice how to go on from that. For example, is SBERT a good choice if most of my queries are in fact multi-sentence paragraphs? I guess what still doesn't add up in my mental model is how people seem to assume that the query embedding vector must somehow automatically be near good answer embedding vecto…

You’re correct. There are lots of ways to project a string of embeddings back onto a single embedding - lots of times however these simpler methods do something like pooling the sum if the word embeddings by averaging and hoping that the answer embedding has a small angle with that projection (cosine similarity). You could try adding a way to focus on individual tokens, pairwise interactions, and more sophisticated ways of pooling and projecting embeddings, and then boom you’ve basically just created transformers.

Re: Introduction to vector similarity search (2022)

#28

Post by freilanzer »

fzliu wrote: post #25
weinzierl wrote: post #24

How would I decide on which granularity level I create my vectors? For example, if I create word level embeddings with word2vec and store one vector per word I probably will only get good results for keyword based queries, right? If I create more fancy vectors, for example on the sentence level, it is not really clear to me how that is supposed to work. Why is the vector embedding of a query, which typically has the…

I would not use word2vec in any application today - I used it in the blog post because it's a well-known model and because the embeddings it generates are static, i.e. one token always maps to the same embedding regardless of context. If you want to create embeddings at the sentence level, a good place to start is SBERT: https://www.sbert.net/

> I would not use word2vec in any application today

Sbert needs to be finetuned to get anwhere near good results if the training objective deviates from the original one. Word2Vec can be estimated from collections of tokens in an unsupervised fashion, so it definitely has its place even today.

Re: Introduction to vector similarity search (2022)

#29

Post by kiwicopple »

computerex wrote: post #5
behnamoh wrote: post #2

Can anyone please suggest a good stack for the following: - calculating text embeddings using open-source/local methods (not OpenAI) - storing them in a vector database. I'm confused by the myriad of options like Chromadb, Pinecone, etc. - running vector similarity search using open-source/local methods. Also, how granular should the text chunks be? Too short and we'll end up with a huge database, too long and we'll…

To calculate embeddings for free, use this very popular model: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... For storing the vectors and doing the vector search: https://github.com/pgvector/pgvector `ankane/pgvector` docker image is a drop in replacement for the postgres image, so you can fire this up with docker very quickly. It's a normal postgres db with a vector datatype. It can index the vector…

(Disclosure: I work at supabase but this applies to any Postgres host that has the pgvector extension)

Here is a guide/notebook using the model above, pgvector, and a python client: https://supabase.com/docs/guides/ai/quickstarts/text-dedupli...

Re: Introduction to vector similarity search (2022)

#30

Post by naraga »

pizza wrote: post #27
weinzierl wrote: post #26

Earlier quoted context omitted.

I understood that it was just an example and it is a good choice for an introductory text. My question is more into advice how to go on from that. For example, is SBERT a good choice if most of my queries are in fact multi-sentence paragraphs? I guess what still doesn't add up in my mental model is how people seem to assume that the query embedding vector must somehow automatically be near good answer embedding vecto…

You’re correct. There are lots of ways to project a string of embeddings back onto a single embedding - lots of times however these simpler methods do something like pooling the sum if the word embeddings by averaging and hoping that the answer embedding has a small angle with that projection (cosine similarity). You could try adding a way to focus on individual tokens, pairwise interactions, and more sophisticated w…

oh boy, this looks like an alchemy to me. on one hand one cant deny success of llm's on another we are shifting the responsibiliteis to non-deterministic fuzzy duck-taped functions.
Post reply on HN