This is incredible! Can't help but wonder if the brain does something similar to fill up "gaps" in reality. e.g how we fill up our perception (not only vision, but general mental intuition) based on just context.
Yes it does.
Deep image prior 'learns' on just one image
81–90 of 235 posts
Re: Deep image prior 'learns' on just one image
#82This is incredible! Can't help but wonder if the brain does something similar to fill up "gaps" in reality. e.g how we fill up our perception (not only vision, but general mental intuition) based on just context.
That's exactly what happens when the retina is damaged. The brain fills in the void imperceptibly so you aren't distracted by the deficit. But I think biology doesn't fill in the void synthetically. It's more likely that the brain "turns a blind eye" toward the missing 'pixels' and directs your attention elsewhere, perhaps to those remaining regions with high saliency (useful detail).
Re: Deep image prior 'learns' on just one image
#83The intel and espionage communities are going to be all over this. This makes the Soviet photo retouching look like child's play. Let's say you want to start a war, and need some evidence of chemical weapons. Now, you can drop in some images of chemical weapons and claim a GAN found them. Sample press releases: "We believe this photo was retouched to hide the chemical weapons. Using a GAN, we recovered clear photogra…
Can't see "we used a GAN" as an explanation that would fly. To counter it you can simply say "well, the GAN could be wrong."
The technique outlined above is useful because we as humans can eyeball the results and say "the improvement on that blurry zebra photo is great!", but we know that it's an improvement based on a networks hueristics that's good enough for us, not an exact replica of the information lost through noise and compression.
Re: Deep image prior 'learns' on just one image
#84Question to anyone that knows this area in depth.
I'm assuming that this impressive feat has a disadvantage (over the traditional example intensive technique of training a CNN with something like ImageNet) in the form of taking a long time to generate the corrected image.
With that assumption in mind, could this new technique be reversed? As in feeding in sharp images, getting back out corrupted ones, for the purposes of generating data sets where there isn't much data to begin with?
You could then take that data and use it to train a more traditional CNN to sort of amortize the results of the technique in the paper and have the process happening faster.
Re: Deep image prior 'learns' on just one image
#85I don't see how any choice of a function g(theta) could have the property they desire, ie could eliminate R(g(theta)). Can anyone explain?
Re: Deep image prior 'learns' on just one image
#86Question to anyone that knows this area in depth. I'm assuming that this impressive feat has a disadvantage (over the traditional example intensive technique of training a CNN with something like ImageNet) in the form of taking a long time to generate the corrected image. With that assumption in mind, could this new technique be reversed? As in feeding in sharp images, getting back out corrupted ones, for the purpose…
Do you mean generative adversarial networks?
https://en.m.wikipedia.org/wiki/Generative_adversarial_netwo...
Re: Deep image prior 'learns' on just one image
#87Earlier quoted context omitted.
I'm wondering the same thing too. Note also that this finding strongly suggests that neural net architecture actually is quite important, possibly even more important than having more data -- which contradicts the conventional wisdom!
There is some pretty strong evidence for this: all the toddlers in the world. You only need to show them something once and they'll immediately be able to recognize more examples of the same thing from different angles and even when it is partially hidden. All they have to guide them is the structure of their brains, not the quantity of data they have been exposed.
"All they have to guide them is the structure of their brains, not the quantity of data they have been exposed."
A typical toddler (say 12 months' old) has spent 4000-5000 hours with open eyes. Even if you assume a low frame rate (10fps), resolution (1080p), and a 1000:1 compression ratio, that's still 1TB of training data.
Re: Deep image prior 'learns' on just one image
#88Earlier quoted context omitted.
> PS. This makes me wonder whether and to what degree the structure of the brain's connectome is a necessary prior for AGI. Well, I wouldn't mix up AGI and AGI by deep learning, and more important I would emphasise that this is a good prior for images . The fundamental insight in CNNs and eventually in this work is that there is a correlation between pairs of nearby pixels. We have something similar for video and aud…
Thanks. I'm not mixing them up! I'm just wondering whether and to what degree architecture , i.e., network structure, will prove important for other, more advanced AI tasks, including up to AGI.
Yes, AGI as a discipline distinguishes itself in large part from machine learning by its focus on architecture over training or data.
Re: Deep image prior 'learns' on just one image
#89I tried to run their ipynb but it wants something called skip. I don't think it's the "skip" on PyPi (that doesn't work anyway) - so what could they be using?
Re: Deep image prior 'learns' on just one image
#90Wow: "In this work, we show that, contrary to expectations, a great deal of image statistics are captured by the structure of a convolutional image generator rather than by any learned capability. This is particularly true for the statistics required to solve various image restoration problems, where the image prior is required to integrate information lost in the degradation processes. To show this, we apply untrain…
I don't understand their editorializing
> contrary to expectations,
ingore the weasel words
> a great deal of image statistics are captured by the structure of a convolutional image generator rather than by any learned capability.
> To show this, we apply untrained ConvNets to the solution of several such problems. Instead of following the common paradigm of training a ConvNet on a large dataset of example images, we fit a generator network to a single degraded image.
How is that _untrained_, if they are training it on an image? Is a generator network different from a ConvNet?
> This is particularly remarkable because no aspect of the network is learned from data;
but they just said "we fit a generator network to a single degraded image"
As others have commented, they appear to be training on all the parts of a single complex image, and using that training to repair local deviations from the global average. This may be better than classical denoising algorithms because the NN can model the image structure better than other approaches, but this doesn't seem like a novel use of NNs.