Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

I think it is our duty to explore all the possibilities of neural networks. They are still not fully understood yet. Theory will catch up once we practically understand the beast. It might not be pure math, but it is a necessary exploration nonetheless.

Matlab seems to need a commercial license for at least one barrier.

Iconic photos are crucial for telling news in a way people remember. For example:

* We remember Tienemen square because of [tank man](https://en.wikipedia.org/wiki/Tank_Man)

* We remember napalm in Vietnam because of [Phan Thi Kim Phuc](https://en.wikipedia.org/wiki/Tank_Man)

* There are too many pictures from the civil rights movement, so I'll [just link to the Getty's gallery](http://www.gettyimages.com/event/the-american-souths-troubli...). These pictures tell the story of violence against black far better than any article ever could.

But there are deeper reasons that mp3 works well, besides "tested against humans". It's just that those reasons are about psychoacoustics. The most obvious example is that humans are bad at discerning amplitude and phase differences at high frequencies.

I expect that as we get deeper into the theory around why particular algorithms work, we'll find similar answers. We'll be able to say things like "when an image has the following statistical properties, we're likely to interpret it as a cat". Of course, right now we can trivially say that if by "the following statistical properties" we mean "activates this specifically trained network in a particular way", but the point is that we can probably boil that down into deeper (and simpler) statements about how human image recognition works.

I can't find a link about it now, but I remember reading that for their Translate app, Google created a way to lossily compress their character recognition neural net. That kind of research seems likely to me to lead to a better understanding of how to boil down the insights that these learning algorithms are gleaning.

This Two Minute Papers video has some thinking along the same lines: https://www.youtube.com/watch?v=ZBWTD2aNb_o

But I'd also point out that the human visual cortex is itself a bunch of models about the real world. It's gotten that way through a combination of evolution and learning. When engineers train neural networks to do tricks like TFA shows, those networks are effectively modeling the real world in two different ways: directly, and by proxy in modeling human visual systems. So the insights we can find by studying them are likely to tell us not only about our own vision, but about the world itself.

Finally, it's worth noting that graphics algorithms have always been about human visual systems and the real physical world, as opposed to deep mathematical truths. Look at this paper for example: http://www.jiansun.org/papers/Dehaze_CVPR2009.pdf

The oversimplified plain English insight those researchers had is "For a photo that doesn't contain haze, you usually can find a fully saturated pixel within some reasonably small neighborhood of any point in the image." There's no deep mathematical insight there, but there is an important insight into the statistical nature of natural images – a fact about the physical world. And they use that insight to great effect: they can remove haze from photographs, and even use the haze as a way of estimating a depth map.

So sure, while you're right that there probably aren't many deep mathematical insights to find from the results of deep learning algorithms (which is generally true of graphics algorithms anyway), I wouldn't gloss over the deep practical insights that they encode.