Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Introduction to vector similarity search (2022)

zilliz.com

Post reply on HN
Print view View original

51–57 of 57 posts

Re: Introduction to vector similarity search (2022)

#51

Post by feoren »

flaghacker wrote: post #50
feoren wrote: post #48

70% of the way through this article they drop "Since we have unit norm vectors ..." and later "Always remember to normalize your embeddings." I found this strange and surprising. It seems to me that in a semantic embedding, "big", "huge", "enormous", and "gargantuan" should roughly point in the same direction but have different magnitudes. For instance, I might assume that the nearest neighbor to "big * 10" is "huge"…

I'm not an expert, but some thoughts: * The problem with large vectors is that they have large dot products with every other vector, which would imply that they are more similar to everything which doesn't make sense. * Adding the requirement that "length==1" doesn't matter much in high-dimensional spaces, since that only removes one degree of freedom. Don't try to use too much 3D intuition here. * It might be intuit…

Very good points here, especially about the fact that the single "length" degree of freedom is much less to lose in very high-dimensional spaces. However, I don't agree that large vectors would end up being "more similar to everything" -- really what's happening is that the dot product stops being a good measure of similarity, but we already knew that using it that way relied on everything being normalized anyway! L1 and L2 still work just fine.

Re: Introduction to vector similarity search (2022)

#53

Post by famouswaffles »

tallytarik wrote: post #47
computerex wrote: post #5

Earlier quoted context omitted.

To calculate embeddings for free, use this very popular model: https://huggingface.co/sentence-transformers/all-MiniLM-L6-v... For storing the vectors and doing the vector search: https://github.com/pgvector/pgvector `ankane/pgvector` docker image is a drop in replacement for the postgres image, so you can fire this up with docker very quickly. It's a normal postgres db with a vector datatype. It can index the vector…

I’ve had some success with this. I think I’d need to fine tune the model to see better results with some domain-specific terms, but I couldn’t find much information about how to actually do that - what sort of input data you need, how much of it, etc. Would be interested to hear if anyone had more to share about fine tuning these models for semantic search.

Instead of fine-tuning, you might try out the Instructor series from here. https://instructor-embedding.github.io/ https://huggingface.co/spaces/mteb/leaderboard

Open source and the way it works is that you give an instruction on the type of task or even domain you want the embedding to be tailored to and embeddings change depending on the instructions.

Re: Introduction to vector similarity search (2022)

#54

Post by famouswaffles »

anon7725 wrote: post #13
NhanH wrote: post #4

Earlier quoted context omitted.

Could you please point to some materials to understand the data needed to train the embedding model for a specific domain?

You don’t need to train anything if you just need embeddings. The data is text. You apply the pretrained model to your text and it returns the embedding. You save it in a vector database if you’re fancy, or a big numpy array if you’re like me. Then run your similarity search (cosine, Euclidean, etc).

a lot of embedding models have poor performance on domain specific data that is only mitigated with finetuning. alternately the instructor series mitigates this by fine-tuning the model on instructions and giving specific instructions to targeted domains.

Re: Introduction to vector similarity search (2022)

#55

Post by famouswaffles »

weinzierl wrote: post #33
naraga wrote: post #30

Earlier quoted context omitted.

oh boy, this looks like an alchemy to me. on one hand one cant deny success of llm's on another we are shifting the responsibiliteis to non-deterministic fuzzy duck-taped functions.

My impression is similar and that's why I asked the question. Another thing that is not clear to me: Is there query directly fed into e.g. SBERT or should I ask an LLM to transform the query into something more suitable, like turning the question into a proposition? Asked more abstractly: In a vector space like SBERT's, can I expect questions and answers about the same topic to lie near each other? Especially will th…

>Another thing that is not clear to me: Is there query directly fed into e.g. SBERT or should I ask an LLM to transform the query into something more suitable, like turning the question into a proposition?

This not how 99% of embedding models work(though you can train for specific tasks) but as it turns out such a thing is possible and is beneficial.

Instruct tuned embedding model here - https://instructor-embedding.github.io/

Re: Introduction to vector similarity search (2022)

#56

Post by pizza »

famouswaffles wrote: post #55
weinzierl wrote: post #33

Earlier quoted context omitted.

My impression is similar and that's why I asked the question. Another thing that is not clear to me: Is there query directly fed into e.g. SBERT or should I ask an LLM to transform the query into something more suitable, like turning the question into a proposition? Asked more abstractly: In a vector space like SBERT's, can I expect questions and answers about the same topic to lie near each other? Especially will th…

>Another thing that is not clear to me: Is there query directly fed into e.g. SBERT or should I ask an LLM to transform the query into something more suitable, like turning the question into a proposition? This not how 99% of embedding models work(though you can train for specific tasks) but as it turns out such a thing is possible and is beneficial. Instruct tuned embedding model here - https://instructor-embedding.…

This kind of thing is going to seem so obvious in hindsight in six months once all the sota methods start converge upon similar improvements lol. It’s still so early, I keep telling myself.

Re: Introduction to vector similarity search (2022)

#57

Post by david_shi »

famouswaffles wrote: post #53
tallytarik wrote: post #47

Earlier quoted context omitted.

I’ve had some success with this. I think I’d need to fine tune the model to see better results with some domain-specific terms, but I couldn’t find much information about how to actually do that - what sort of input data you need, how much of it, etc. Would be interested to hear if anyone had more to share about fine tuning these models for semantic search.

Instead of fine-tuning, you might try out the Instructor series from here. https://instructor-embedding.github.io/ https://huggingface.co/spaces/mteb/leaderboard Open source and the way it works is that you give an instruction on the type of task or even domain you want the embedding to be tailored to and embeddings change depending on the instructions.

Instructor performs very well but is still limited to its training corpus. For niche domains, fine-tuning can be helpful in improving performance.
Post reply on HN