Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Blade Runner re-encoded using neural networks

vox.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: Blade Runner re-encoded using neural networks

#11

Post by mcv »

dogma1138 wrote: post #3

Can anyone explain what does this encoder actually does and how is it different than any other encoder? The article almost seems to make it sound that this encoder somehow reconstructed the film...

As far as I can tell, it just sent it through a deep learning network. Cool, but not exactly the same as an AI reconstructing the movie. It's just a distorted version of the original, and Warner Bros was not entirely incorrect in requesting its removal (though it doesn't hurt them much to allow it, because it's so heavily distorted).

Re: Blade Runner re-encoded using neural networks

#12

Post by kastnerkyle »

manmal wrote: post #6
kastnerkyle wrote: post #5

Earlier quoted context omitted.

This technique is quite different from deep dream. An autoencoder is a model that is trained to compress then reconstruct an image. After seeing and trying to reconstruct many images, it learns how to make a "compressed representation" from original images by going through this compress -> uncompress scheme, which typically results in blurry/lossy reconstructions of images which are very different than the normal art…

Would it be possible to make it look better or even the same by increasing digits per frame (eg 1000 instead of 200)?

Yes, although the particular technique used here (variational autoencoder, or VAE) doesn't benefit from increasing the code space due to a particular penalty (KL divergence against a fixed N(0, 1) Gaussian prior) during training. So even by making it go to 1000 digits in the code space, the model will probably still only choose to use 5 or 10 dimensions unless the KL penalty is relaxed, making it closer to a standard autoencoder.

An autoencoder with a much larger code space could be an improvement, or some newer works such as Adversarially Learned Inference / Adversarial Feature Learning [0, 1] or real NVP [2, 3] could probably do a much better job at the task, at the cost of increased computation.

Also something like inpainting parts of the frames with pixel RNN [4] would be interesting.

[0] http://arxiv.org/abs/1606.00704

[1] http://arxiv.org/abs/1605.09782

[2] http://www-etud.iro.umontreal.ca/~dinhlaur/real_nvp_visual/

[3] http://arxiv.org/abs/1605.08803

[4] http://arxiv.org/abs/1601.06759

Re: Blade Runner re-encoded using neural networks

#16

Post by IshKebab »

mcv wrote: post #11
dogma1138 wrote: post #3

Can anyone explain what does this encoder actually does and how is it different than any other encoder? The article almost seems to make it sound that this encoder somehow reconstructed the film...

As far as I can tell, it just sent it through a deep learning network. Cool, but not exactly the same as an AI reconstructing the movie. It's just a distorted version of the original, and Warner Bros was not entirely incorrect in requesting its removal (though it doesn't hurt them much to allow it, because it's so heavily distorted).

Autoencoders are a special type of deep neural network where there is a small middle layer, and the labels are the same as the input.

The idea is you train the network to reproduce the input, but it has to pass the information through the small middle layer which has relatively few neurons. Thus the network learns a good (hopefully) way to represent typical inputs with not much information. You have automatically generated an encoding (hence the name).

You're right it's not really 'reconstructing' the film in the way they imply. But it isn't just a distorted version of the original either. The really important question is how big the middle layer is (and hence how much it has learned / how good the compression is). Unless this is just done for artistic purposes, which seems to be the case.

Re: Blade Runner re-encoded using neural networks

#18

Post by kefka »

timlyo wrote: post #13

It's saying down for scheduled maintenance for me, anyone got a mirror?

Sure do.

If you're running IPFS locally use this link: http://127.0.0.1:8080/ipfs/QmPo9savMewcGCzUhzNQi96b6gSSoQ4k4...

If not, use this one (public IPFS gateway) http://ipfs.io/ipfs/QmPo9savMewcGCzUhzNQi96b6gSSoQ4k4DFEVkF5...

Be aware, there is no sound with this video. You'll have to add it from your own sources (nor did I want to add the sound from my dvd set).

And frankly, IPFS is awesome :)

Re: Blade Runner re-encoded using neural networks

#19

Post by vintermann »

kastnerkyle wrote: post #12
manmal wrote: post #6

Earlier quoted context omitted.

Would it be possible to make it look better or even the same by increasing digits per frame (eg 1000 instead of 200)?

Yes, although the particular technique used here (variational autoencoder, or VAE) doesn't benefit from increasing the code space due to a particular penalty (KL divergence against a fixed N(0, 1) Gaussian prior) during training. So even by making it go to 1000 digits in the code space, the model will probably still only choose to use 5 or 10 dimensions unless the KL penalty is relaxed, making it closer to a standard…

Given that Adversarially Learned Inference came out yesterday, the best approach may be to just wait a couple of months and see what the state of the art is then.

Re: Blade Runner re-encoded using neural networks

#20

Post by mhax »

This reminds me very much of Parag Mital's work on audiovisual resynthesis: http://pkmital.com/home/

IIRC he has been beset with takedown notices for his 'smash up' videos which resynthesise the weeks #1 most viewed youtube video from the next 9 most viewed videos http://pkmital.com/home/youtube-smash-up/

Post reply on HN