Deep image prior 'learns' on just one image

  min_x E(x; x0) + R(x)

I thought that E(x; x0) would denote the error/difference between original and corrupted images, and R(x) be the (searched-for) correction. But this doesn't seem to make sense with the next parts of their explanation.

>The reason for this is because in the intervening years we gather more training data, better training techniques, and more computational power. So we can instead use a more homogeneous architecture which has _less_ assumptions (less priors) and at the end of the day we get _better_ results.

>I'll repeat that. We put _less_ priors into our networks and we get better results.

Is move from VGG-like architectures a question of moving to using less priors or just move to different priors that work better?

In addition, I'd claim that a choice of training method also imbues the whole ML system with prior knowledge. If we add a dropout layer, we wish to enforce a particular regularization scheme on the model because we know that we need to regularize to model to avoid overfitting. If we choose to use a particular optimizer instead of another, we use it because we know it has desirable properties on particular objective objection.

>So we already know that architecture isn't ultimately important. You can have a giant, fully connected network, and it _will_ work, if you feed it enough data and have the computational power necessary to train such a beast.

I'm slightly unclear about your argument here. What is the point of training the fully connected network for a long time and with massive datasets to get equivalent results and structure as convnets if we already do know that convolutional structure with weight sharing is a good fit for natural images? Deep learning has been casted as a magical black box that produces impressive results "yet the scientists don't know why!" by the press. On the contrary, these results suggest there might be less magic in neural net models as the recent hype would suggest.

> The examples are clearly lab queens, where the occluded regions are not particularly interesting/challenging.

To me they look quite like the standard examples used in inpainting and denoising articles.

I do not think intelligence is a consequence of structure. Transparency is a consequence of how light interacts with objects. There is no "transparent gold". And being transparent is not something we can program gold to do.

Programming is the application of an electric field across a silicon surface: this can no more transmute silicon into gold as it can into a nervous system.

Consciousness is a biological activity. It is not something wood, silicon, sand, metal, glass, water, etc. can do by virtue of moving around in an interesting pattern. To me, this is akin to believing magic, or to thinking that the issue with Bug's Bunny's consciousness is that the ink wasnt laid out in the right way. Rather, there isnt any.

As scientists we should begin very clearly by identifying the target phenomenon. Line up everything in the known universe that is conscious and you will find that it engages in a highly specific biochemical activity that requires a whole set of highly specific chemical interactions.

To believe these can be achieved by applying an electric field across some silicon seems like the victorian superstition that a person could be raised from the dead by doing likewise.

Rather, all we are able to do is imitate in the weakest sense. People fooled by chatbots are no more enlightened than the early victorians who were shocked to watch the movie of a train coming at them. Fooling people is a very easy thing to do. And esp. in the area of consciousness when "alertness to things that might be conscious" seems a deep psychological preoccupation of almost every animal to the point of absurdity (eg. being scared at the wind).