Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Supercharge vector search with ColBERT rerank in PostgreSQL

blog.vectorchord.ai

Post reply on HN
Print view View original

11–15 of 15 posts

Re: Supercharge vector search with ColBERT rerank in PostgreSQL

#11

Post by upghost »

tadkar wrote: post #5
simonw wrote: post #3

> However, generating sentence embeddings through pooling token embeddings can potentially sacrifice fine-grained details present at the token level. ColBERT overcomes this by representing text as token-level multi-vectors rather than a single, aggregated vector. This approach, leveraging contextual late interaction at the token level, allows ColBERT to retain more nuanced information and improve search accuracy comp…

Here’s my understanding. It is intimidating to write a response to you, because you have an exceptionally clear writing style. I hope the more knowledgable HN crowd will correct any errors in fact or presentation style below. Old school word embedding models (like Word2Vec) come up with embeddings by using masked word predictions. You can embed a whole sentence by taking the average of all the word embeddings in the…

If this is true, my understanding of vanilla token vector embeddings is wrong. my understanding was that the vector embedding was the geometric coordinates of the token in the latent space with respect to the prior distribution. So adding another dimension to make it a "multivector" doesn't (in my mind) seem like it would add much. What am I missing?

Re: Supercharge vector search with ColBERT rerank in PostgreSQL

#12

Post by lysecret »

nighthawk454 wrote: post #8
simonw wrote: post #3

> However, generating sentence embeddings through pooling token embeddings can potentially sacrifice fine-grained details present at the token level. ColBERT overcomes this by representing text as token-level multi-vectors rather than a single, aggregated vector. This approach, leveraging contextual late interaction at the token level, allows ColBERT to retain more nuanced information and improve search accuracy comp…

Usually, we destructively compress (mean-pooling) both the query and the document, and then compare the two compressed forms. With ColBERT, we compare first - at the full token level - for more detailed comparison. Then reduce the full set of comparisons to a single vector. Naturally this takes more memory and compute to do the more comparisons. The idea is it’s worth it because the more detailed comparisons lead to…

That’s actually a very good explanation thanks!

Re: Supercharge vector search with ColBERT rerank in PostgreSQL

#13

Post by byefruit »

simonw wrote: post #3

> However, generating sentence embeddings through pooling token embeddings can potentially sacrifice fine-grained details present at the token level. ColBERT overcomes this by representing text as token-level multi-vectors rather than a single, aggregated vector. This approach, leveraging contextual late interaction at the token level, allows ColBERT to retain more nuanced information and improve search accuracy comp…

What I don't understand is how you don't end up with a totally impractical number of vectors if you have one per token? Surely nobody is storing that many in any real system?

Re: Supercharge vector search with ColBERT rerank in PostgreSQL

#14

Post by tadkar »

upghost wrote: post #11
tadkar wrote: post #5

Earlier quoted context omitted.

Here’s my understanding. It is intimidating to write a response to you, because you have an exceptionally clear writing style. I hope the more knowledgable HN crowd will correct any errors in fact or presentation style below. Old school word embedding models (like Word2Vec) come up with embeddings by using masked word predictions. You can embed a whole sentence by taking the average of all the word embeddings in the…

If this is true, my understanding of vanilla token vector embeddings is wrong. my understanding was that the vector embedding was the geometric coordinates of the token in the latent space with respect to the prior distribution. So adding another dimension to make it a "multivector" doesn't (in my mind) seem like it would add much. What am I missing?

I think the important thing is that the first approach to converting complete sentences to an embedding was done by averaging all the embeddings of the tokens in the sentence. What ColBERT does is store the embeddings of all the tokens before then using dot products to identify the most relevant tokens to the query. Another comment in this thread says the same thing in a different way. Feels funny to post a stack exchange reference, but this is a great answer!

[1] https://stackoverflow.com/questions/57960995/how-are-the-tok...

Post reply on HN