Contrastive Self-Supervised Learning
ankeshanand.com
Contrastive Self-Supervised Learning
1–10 of 14 posts
Re: Contrastive Self-Supervised Learning
#2I hadn’t heard of this before. Cool. Going to share this with my team on Monday.
Re: Contrastive Self-Supervised Learning
#3Great write up, I especially liked the section of Contrastive Predictive Coding, I think that's going to be the next iteration of ML.
Re: Contrastive Self-Supervised Learning
#4There's a lot to like in this article, but I don't quite agree with the setup. I think it's better to think of "contrastive" approaches as being orthogonal to basic self-supervised learning methods - they represent an additional piece you can add to your loss function that results in very significant improvements. This approach can be combined with existing self-supervised pretext tasks.
I've discussed these ideas here, for those that are interested in learning more: https://www.fast.ai/2020/01/13/self_supervised/
Re: Contrastive Self-Supervised Learning
#5There's a lot to like in this article, but I don't quite agree with the setup. I think it's better to think of "contrastive" approaches as being orthogonal to basic self-supervised learning methods - they represent an additional piece you can add to your loss function that results in very significant improvements. This approach can be combined with existing self-supervised pretext tasks. I've discussed these ideas he…
I didn't mean to convey that we should abandon generative self-supervised methods, but I can see how comparing them gives that impression.
Agree that using them in conjunction would make sense, since generative methods could capture some features better and vice versa.
Re: Contrastive Self-Supervised Learning
#6Great write up, I especially liked the section of Contrastive Predictive Coding, I think that's going to be the next iteration of ML.
What’s the current iteration of ML?
Re: Contrastive Self-Supervised Learning
#7Re: Contrastive Self-Supervised Learning
#8Re: Contrastive Self-Supervised Learning
#9There's a lot to like in this article, but I don't quite agree with the setup. I think it's better to think of "contrastive" approaches as being orthogonal to basic self-supervised learning methods - they represent an additional piece you can add to your loss function that results in very significant improvements. This approach can be combined with existing self-supervised pretext tasks. I've discussed these ideas he…
BTW, one thing which makes it a bit hard to get into self-supervised learning is that the most common benchmarking task involves pretraining on Imagenet, which is too slow and expensive for development.
I recently created a little dataset that is specifically designed to allow for testing out self-supervised techniques, called Image网 ("Imagewang"). I'd love to see some folks try it out, and submit strong baselines to the leaderboard: https://github.com/fastai/imagenette#image%E7%BD%91
Re: Contrastive Self-Supervised Learning
#10There's a lot to like in this article, but I don't quite agree with the setup. I think it's better to think of "contrastive" approaches as being orthogonal to basic self-supervised learning methods - they represent an additional piece you can add to your loss function that results in very significant improvements. This approach can be combined with existing self-supervised pretext tasks. I've discussed these ideas he…
BTW, one thing which makes it a bit hard to get into self-supervised learning is that the most common benchmarking task involves pretraining on Imagenet, which is too slow and expensive for development. I recently created a little dataset that is specifically designed to allow for testing out self-supervised techniques, called Image网 ("Imagewang"). I'd love to see some folks try it out, and submit strong baselines to…
I might take you up on that. How does your dataset facilitate self supervised experimentation?
I've a good amount of experience playing with autoencoders but this is the first I've heard of contrastive learning.