It seems that this subpixel convolution layer is equivalent to what is known in the neural net community as the "deconvolution layer" but it is much more memory and computation efficient. The interlacing rainbow picture was a bit hard to understand until I read this https://export.arxiv.org/ftp/arxiv/papers/1609/1609.07009.pd...
Subpixel: A subpixel convolutional neural network implementation with Tensorflow
21–30 of 47 posts
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#22I'm not sure, but there seems to be something wonky in the input images. They are very blocky, so I thought that they would be just pixel doubled (or quadrupled) from low-res pictures, but the blockiness lacks the regularity I'd expect from pixel-doubled images. How were the input images prepared?
Super wonky indeed. Also it should compare to something like photoshops bicubic enlargement or the original size, because the brain gets stuck on the pixel edges.
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#23The explanation in the README of the github project is excellent and well-written! Here's a really great set of animations by Vincent Dumoulin on how various conv operators work: https://github.com/vdumoulin/conv_arithmetic
And https://arxiv.org/abs/1603.07285 for the corresponding paper. Really clear and easy-to-understand explanation of some of the math.
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#24I'm not sure, but there seems to be something wonky in the input images. They are very blocky, so I thought that they would be just pixel doubled (or quadrupled) from low-res pictures, but the blockiness lacks the regularity I'd expect from pixel-doubled images. How were the input images prepared?
Super wonky indeed. Also it should compare to something like photoshops bicubic enlargement or the original size, because the brain gets stuck on the pixel edges.
If you are interested in how it compares to bicubic or the original. Check these papers using the sub pixel convolutional layer: https://arxiv.org/abs/1609.05158 https://arxiv.org/abs/1609.04802.
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#25And I always wondered how those photo enhancers in Blade Runner worked...!
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#26So, basically, this is the thing in a crime detective movie where the forensic analyst is looking at a terrible pixelated surveillance camera still and says "enhance," and the computer magically increases the resolution to reveal the culprit's face. Just another entry on the "things that are supposed to be impossible that convolutional nets can do now."
yup, to certain point! there are information theoretic limits though. You can fill in information, but there will be biases to a certain point. in this case defined by the dataset. if the "enhance" is too strong, we should be careful with what we do with the results in forensics. but man, it can make your internet pics look smooth! :) thanks for the comment!
This (overenhancement) was a minor plot point in Crichton's novel Congo, IIRC.
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#27So, basically, this is the thing in a crime detective movie where the forensic analyst is looking at a terrible pixelated surveillance camera still and says "enhance," and the computer magically increases the resolution to reveal the culprit's face. Just another entry on the "things that are supposed to be impossible that convolutional nets can do now."
Here is the ridiculous Let's Enhance supercut that all just became realistic: https://www.youtube.com/watch?v=LhF_56SxrGk
(Created by the super talented duncanrobson)
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#28This is impressive! But, I'll be really impressed once this 'new thing' brings us roto masks in motion. That is, isolating objects from background on a movie with pixel-perfect accuracy. It will also make a lot of people out of job and a lot of people happy at the same time.
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#29Interesting image "upscale" algorithm. I'm not familiar enough with the field to understand how the "neutral net" part feeds in, other than to do parallel computation on the x-pos, y-pos, (RGB) color-type-intensity tensor interpolated/weighted into a larger/finer tensor. (linear algebra speak for upscaling my old DVD to HD, that sort of thing) At the risk of exposing my ignorance, this has nothing to do with "AI", ri…
Everything that we understand how to do is "not really AI". It's only "AI" when it's still a mystery. At least that's the way people act.
Re: Subpixel: A subpixel convolutional neural network implementation with Tensorflow
#30This is impressive! But, I'll be really impressed once this 'new thing' brings us roto masks in motion. That is, isolating objects from background on a movie with pixel-perfect accuracy. It will also make a lot of people out of job and a lot of people happy at the same time.
Considering motion blur, "pixel-perfect" is a difficult requirement.