91–100 of 235 posts
The idea of not running the simulation `past` the realistic interpretation and using that result makes sense but the results are way beyond what I would have expected!
Great work on the write up.
I wonder if you could do something similar with audio
Wow, so just the weight sharing architecture does so much already? I am wondering if the same could be done with LSTMs on sequences or CNNs on voice...
I'm wondering the same thing too. Note also that this finding strongly suggests that neural net architecture actually is quite important, possibly even more important than having more data -- which contradicts the conventional wisdom!
So what makes this result possible is not the architecture of NN in CNN but rather architecture of C. That allows us to get multiple samples from single image. The rest is just that actual content of the image is easier to learn then the noise.
Brain is almost nothing like CNN.
Earlier quoted context omitted.
The “task dependent observation model” also comes out of thin air, or is it trained with lots of data?
Observation model is trivial and data-free. For example, just mean squared error for denoising, mean squared error of downscaled image for super-resolution, etc.
Wow: "In this work, we show that, contrary to expectations, a great deal of image statistics are captured by the structure of a convolutional image generator rather than by any learned capability. This is particularly true for the statistics required to solve various image restoration problems, where the image prior is required to integrate information lost in the degradation processes. To show this, we apply untrain…
I don't understand their editorializing > contrary to expectations, ingore the weasel words > a great deal of image statistics are captured by the structure of a convolutional image generator rather than by any learned capability. > To show this, we apply untrained ConvNets to the solution of several such problems. Instead of following the common paradigm of training a ConvNet on a large dataset of example images, we…
Earlier quoted context omitted.
Observation model is trivial and data-free. For example, just mean squared error for denoising, mean squared error of downscaled image for super-resolution, etc.
I'm fairly certain that's not? Where in the paper does it talk about this other network, generally called the discriminator?
I tried to run their ipynb but it wants something called skip. I don't think it's the "skip" on PyPi (that doesn't work anyway) - so what could they be using?
Earlier quoted context omitted.
I'm wondering the same thing too. Note also that this finding strongly suggests that neural net architecture actually is quite important, possibly even more important than having more data -- which contradicts the conventional wisdom!
There is some pretty strong evidence for this: all the toddlers in the world. You only need to show them something once and they'll immediately be able to recognize more examples of the same thing from different angles and even when it is partially hidden. All they have to guide them is the structure of their brains, not the quantity of data they have been exposed.
Earlier quoted context omitted.
There is some pretty strong evidence for this: all the toddlers in the world. You only need to show them something once and they'll immediately be able to recognize more examples of the same thing from different angles and even when it is partially hidden. All they have to guide them is the structure of their brains, not the quantity of data they have been exposed.
"All they have to guide them is the structure of their brains, not the quantity of data they have been exposed." A typical toddler (say 12 months' old) has spent 4000-5000 hours with open eyes. Even if you assume a low frame rate (10fps), resolution (1080p), and a 1000:1 compression ratio, that's still 1TB of training data.
I don't see how any choice of a function g(theta) could have the property they desire, ie could eliminate R(g(theta)). Can anyone explain?