Self-Supervised Tracking via Video Colorization
ai.googleblog.com
Self-Supervised Tracking via Video Colorization
1–10 of 17 posts
Re: Self-Supervised Tracking via Video Colorization
#2Very clever, and "obvious" only in hindsight: Training a deep convnet to colorize all frames in a grayscale video clip from a single color frame taken from the same clip induces the neural net to learn to track all objects in the video, with robustness to occlusions, change of viewing angles, etc. Labels are not required; only a color frame from each clip. Most impressively, the embeddings learned by the convnet (i.e., the representations learned by the next-to-last layer) are linearly separable by object. Very nice!
Re: Self-Supervised Tracking via Video Colorization
#3I saw something similar before (from gifs.com's sticker editor of all things lol) where you annotate the segmentation of the first frame of the video and it will propagate that segmentation to the rest of the frames:
https://medium.com/gifs-ai/interactive-segmentation-with-con...
Re: Self-Supervised Tracking via Video Colorization
#4I'm very skeptical that there is any merit to the 'tracking' over other techniques, as well as the colorization being better than this 14 year old paper:
http://webee.technion.ac.il/people/anat.levin/papers/coloriz...
The results in their videos look very poor.
Re: Self-Supervised Tracking via Video Colorization
#5I'm very skeptical that there is any merit to the 'tracking' over other techniques, as well as the colorization being better than this 14 year old paper: http://webee.technion.ac.il/people/anat.levin/papers/coloriz... The results in their videos look very poor.
Tracking is not the method they're using for colourizing, but the other way around. Your linked paper has no tracking.
Re: Self-Supervised Tracking via Video Colorization
#6Very clever, and "obvious" only in hindsight: Training a deep convnet to colorize all frames in a grayscale video clip from a single color frame taken from the same clip induces the neural net to learn to track all objects in the video, with robustness to occlusions, change of viewing angles, etc. Labels are not required; only a color frame from each clip. Most impressively, the embeddings learned by the convnet (i.e…
Simple to explain, yes, but I feel like this isn't even really "obvious" even in hindsight. This whole thing is very clever.
The only complaint I have is that it's not better than supervised object tracking, so I wonder if this idea is too late?
To draw a parallel to image classification, at one point in time neural nets were trained with a bunch of unsupervised pre-training using reconstruction loss, but that technique has basically fallen by the wayside as we've gotten larger datasets and found a pile of tricks for training them from scratch.
Re: Self-Supervised Tracking via Video Colorization
#7Very clever, and "obvious" only in hindsight: Training a deep convnet to colorize all frames in a grayscale video clip from a single color frame taken from the same clip induces the neural net to learn to track all objects in the video, with robustness to occlusions, change of viewing angles, etc. Labels are not required; only a color frame from each clip. Most impressively, the embeddings learned by the convnet (i.e…
Simple to explain, yes, but I feel like this isn't even really "obvious" even in hindsight. This whole thing is very clever. The only complaint I have is that it's not better than supervised object tracking, so I wonder if this idea is too late? To draw a parallel to image classification, at one point in time neural nets were trained with a bunch of unsupervised pre-training using reconstruction loss, but that techni…
Labeling object locations in all frames of a large number of video clips is significantly more expensive than labeling a comparably large number of images.
Re: Self-Supervised Tracking via Video Colorization
#8I'm very skeptical that there is any merit to the 'tracking' over other techniques, as well as the colorization being better than this 14 year old paper: http://webee.technion.ac.il/people/anat.levin/papers/coloriz... The results in their videos look very poor.
Tracking is not the method they're using for colourizing, but the other way around. Your linked paper has no tracking.
I realize that.
They are for some reason saying they can track things with their colorization, when their colorization is extremely unimpressive as well as the tracking that results from using it.
There is no reason colorization needs to happen to do the tracking anyway. The tracking is unimpressive and now indirect.
This isn't some sort of epiphany they've discovered, they are just reinventing video image segmentation very poorly.
Here are half a dozen examples from a 30 second google search:
https://www.youtube.com/watch?v=juDvLrFQF0U
https://www.youtube.com/watch?v=JYgyDdLf7GQ
https://static.googleusercontent.com/media/research.google.c...
https://perso.liris.cnrs.fr/nicolas.bonneel/InteractiveMulti...
http://files.is.tue.mpg.de/black/papers/TsaiCVPR2016.pdf
https://graphics.ethz.ch/~perazzif/bvs/files/bvs.pdf
The only reason this is news is because it's google and the researchers seem to think they've discovered something. Techniques like this with much better results have been shown at Siggraph for decades.
Re: Self-Supervised Tracking via Video Colorization
#9Earlier quoted context omitted.
Simple to explain, yes, but I feel like this isn't even really "obvious" even in hindsight. This whole thing is very clever. The only complaint I have is that it's not better than supervised object tracking, so I wonder if this idea is too late? To draw a parallel to image classification, at one point in time neural nets were trained with a bunch of unsupervised pre-training using reconstruction loss, but that techni…
Labeling object locations in all frames of a large number of video clips is significantly more expensive than labeling a comparably large number of images.
Sure, but like Imagenet, these datasets already exist. So unless these models are quite brittle to the objects being tracked, this is likely not going to be an issue.
Re: Self-Supervised Tracking via Video Colorization
#10Earlier quoted context omitted.
Tracking is not the method they're using for colourizing, but the other way around. Your linked paper has no tracking.
I realize that. They are for some reason saying they can track things with their colorization, when their colorization is extremely unimpressive as well as the tracking that results from using it. There is no reason colorization needs to happen to do the tracking anyway. The tracking is unimpressive and now indirect. This isn't some sort of epiphany they've discovered, they are just reinventing video image segmentati…
Right, I totally agree. What is more, I think this same result could actually be sold without the pompous deep learning bullshit and be received quite differently. If they did not claim to invent the wheel, but rather modestly noted their observation (which in a limited way is actually quite cool - that is from a known - at least on this forum - deep learning skeptic like me), it would make a much better impression.
Same is true actually for many DL papers. They'd be actually cool, if they weren't oversold.