Similarity Learning lacks a framework. So we built one • Hacker News Board

Re: Similarity Learning lacks a framework. So we built one

#21

Post by monatis » Wed, Jul 13, 2022, 6:29 PM UTC

usgroup wrote: post #20

I’m familiar with metric learning within the Mahalanobis family for kNN oriented applications . I’m not getting what use cases this framework targets? Is it custom image search type stuff which may benefit from fine tuning? What is a realistic minimum viable dataset for an approach like this? When is it not advisable? How does it compare to other more basic approaches?

The main idea is to train a deep learning model to encode a high-dimensional sample to a low-dimensional vector in a latent space. Then it can be used in various downstream tasks such as KNN applications, semantic search, multimodal retrieval, recommendation systems, anomaly detection etc. It's not limited to the image domain --it can be also audios, texts, videos, or more specific entities such as authors, soccer players, songs etc. The size of the dataset can be thought of being similar to other deep learning methods, but you can make a choice among various similarity learning methods based on the size of your dataset or according to whether it's labeled or not. A common approach is (1) to train a base model by using a self-supervised method with a bulk amount of unlabeled data and (2) to finetune it on a more specific domain or task with a smaller labeled dataset. If you can start with a pretrained model such as ResNet or BERT, you can skip the first step.

Re: Similarity Learning lacks a framework. So we built one

#22

Post by usgroup » Wed, Jul 13, 2022, 6:53 PM UTC

usgroup wrote: post #20

I’m familiar with metric learning within the Mahalanobis family for kNN oriented applications . I’m not getting what use cases this framework targets? Is it custom image search type stuff which may benefit from fine tuning? What is a realistic minimum viable dataset for an approach like this? When is it not advisable? How does it compare to other more basic approaches?

To answer my own question:

https://qdrant.tech/

Re: Similarity Learning lacks a framework. So we built one

#23

Post by usgroup » Wed, Jul 13, 2022, 6:58 PM UTC

monatis wrote: post #21

usgroup wrote: post #20

I’m familiar with metric learning within the Mahalanobis family for kNN oriented applications . I’m not getting what use cases this framework targets? Is it custom image search type stuff which may benefit from fine tuning? What is a realistic minimum viable dataset for an approach like this? When is it not advisable? How does it compare to other more basic approaches?

The main idea is to train a deep learning model to encode a high-dimensional sample to a low-dimensional vector in a latent space. Then it can be used in various downstream tasks such as KNN applications, semantic search, multimodal retrieval, recommendation systems, anomaly detection etc. It's not limited to the image domain --it can be also audios, texts, videos, or more specific entities such as authors, soccer pl…

I’d be surprised if it’d be useful for something like cars or soccer players, or really anything that may not have a continuous mapping. I guess more generally whenever the underlying “true” similarity function is not differentiable — categorical data springs to mind (cars, football players…).

I could see it making sense for complex unstructured data — Qdrant seems to point in that direction.

Re: Similarity Learning lacks a framework. So we built one

#24

Post by monatis » Wed, Jul 13, 2022, 7:25 PM UTC

usgroup wrote: post #23

monatis wrote: post #21

Earlier quoted context omitted.

The main idea is to train a deep learning model to encode a high-dimensional sample to a low-dimensional vector in a latent space. Then it can be used in various downstream tasks such as KNN applications, semantic search, multimodal retrieval, recommendation systems, anomaly detection etc. It's not limited to the image domain --it can be also audios, texts, videos, or more specific entities such as authors, soccer pl…

I’d be surprised if it’d be useful for something like cars or soccer players, or really anything that may not have a continuous mapping. I guess more generally whenever the underlying “true” similarity function is not differentiable — categorical data springs to mind (cars, football players…). I could see it making sense for complex unstructured data — Qdrant seems to point in that direction.

Some loss functions such as ArcFace loss and CosFace loss enforce the encoder model to organize their latent space in such a way that categories are placed with an angular margin from one another. Thus the model implicitly learns a continuous distance function.

Fun fact, one of the examples in Quaterion is for similar cars search.

If you find this topic and want to discover more, we collected a bunch of resources that might be helpful. https://github.com/qdrant/awesome-metric-learning

Re: Similarity Learning lacks a framework. So we built one

#25

Post by fzliu » Wed, Jul 13, 2022, 8:58 PM UTC

Great article. I've been working in and around this space since 2014, and I think similarity learning, vector search, and embedding management will be a core part of future applications that leverage ML.

I recently built a similarity search application that recommends new Pinterest users channels to follow based on liked images using Milvus (https://github.com/milvus-io/milvus) as a backend. Similarity learning is a huge part of it, and I'm glad more and more tools like Quaterion are being released to help make this kind of tech ubiquitous.