Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Autoencoding Blade Runner: reconstructing films with artificial neural networks

medium.com

Post reply on HN
Print view View original

11–20 of 39 posts

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#11

Post by aab0 »

bcheung wrote: post #3

Very interesting. This makes me wonder if a similar technique can be used for compression?

Absolutely. The point of an autoencoder is dimensionality reduction: boil a big set of data down to a few hundred or thousand numbers in a vector which summarizes it. You could treat it either as lossy compression and store just the encoding, or you can treat it as a hybrid format in which the autoencoder lossy encoding is then corrected to lossless by additional bits in the stream.

In practice, even the hyper-efficient compression algorithms used in something like zpaq tend to use only very small shallow predictive neural networks because no one wants to wait days for their data to be compressed or ship around big neural nets as part of their archives, so it's more of an information-theoretic curiosity. Few enough people will even use 'xz'.

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#12

Post by pvg »

Shivetya wrote: post #5

on a side note, I never heard this voice over version before. I am only used to the Harrison Ford voice over. This one lets me understand why many didn't like the VO. Now back to the article, can someone explain about how many passes before it gets to near film quality? Can it extrapolate missing frames eventually?

That particular voiceover is from the theatrical trailer, it wasn't in any of the 125050123.07 versions of the movie. You can see it here:

https://www.youtube.com/watch?v=4lW0F1sccqk

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#13

Post by csn »

bcheung wrote: post #3

Very interesting. This makes me wonder if a similar technique can be used for compression?

I was intrigued by this a while back. I think of training a NN as generating a function, an equation, from a training set which, given a specific input, outputs a prediction. If you can come up with an equation-input pair that, when executed, a) accurately enough approximates some data, and b) requires less space than the original file, you have achieved (most likely lossy) compression.

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#15

Post by visarga »

aardshark wrote: post #4

What is the difference between using a neural network to do this and using a filter that obtains the same or similar effect by distorting the frames of the input randomly? I guess I feel like there's no practical result here. It's only interesting from an aesthetic point of view. Am I being unfair?

With this network you can re-encode the movie to make everyone smiling and wearing glasses. You can't do that with simple distortions.

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#16

Post by visarga »

joe_the_user wrote: post #8
aardshark wrote: post #4

What is the difference between using a neural network to do this and using a filter that obtains the same or similar effect by distorting the frames of the input randomly? I guess I feel like there's no practical result here. It's only interesting from an aesthetic point of view. Am I being unfair?

After reading the article, I'm still not sure what the purpose of the training is. If they're trying to reconstruct a film from stills, it seems like a failure since it looks like they wind-up with all sorts of swirly stuff rather than, say, the original film. If they're trying to create interesting swirly stuff, where do they intend to go after that? I mean, sure it's aesthetic though not on the level of weirdness o…

I am not an expert, but: This approach creates powerful embeddings for images. It can convert an image into embedding space and vice-versa, generate images back from embeddings. It is built to function like a perception and imagination module. The embeddings are much lower dimensional and the latent variables are disentangled. There is a component for "has glasses" which you can flip and get the same image + glasses, for example. It is obvious this would be very useful in building all sorts of classifiers, image generators and agents (because agents need to compute reward over state and action space and disentangled representations of the state space are good for this task).

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#17

Post by nathancahill »

xt00 wrote: post #9

It's a cool thing this guy did.. It would be interesting to see how small the files are that are generated in this process.. Just low pass filtering the video like somebody else suggested would probably achieve a similarly lossy image. I guess what I take away from this is that maybe the way we store info in our brains looks kind of like this? Kinda fuzzy versions of the real thing? Would be interesting if somebody o…

2011: http://news.berkeley.edu/2011/09/22/brain-movies/

There's probably newer research out there too.

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#18

Post by chillacy »

argonaut wrote: post #7

Correct me if I haven't looked into this closely, but one glaring problem is that all the results are from the training set. So it's not surprising you get something movie-ish by running the network over a movie it was trained on ; the network has already seen what the output of the movie should look like.

Seems like his goal is regenerating the training set (with lossy recall) rather than predicting never before seen frames. When I hear old songs I haven't listened to in years I can still remember the next line, just based on the previous line. That's a lot of data that's being encoded somehow in my brain.

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#19

Post by vintermann »

aab0 wrote: post #11
bcheung wrote: post #3

Very interesting. This makes me wonder if a similar technique can be used for compression?

Absolutely. The point of an autoencoder is dimensionality reduction: boil a big set of data down to a few hundred or thousand numbers in a vector which summarizes it. You could treat it either as lossy compression and store just the encoding, or you can treat it as a hybrid format in which the autoencoder lossy encoding is then corrected to lossless by additional bits in the stream. In practice, even the hyper-effici…

You could, but I don't think this would be competitive on compression ratio at all, even allowing for an order of magnitude more time.

Re: Autoencoding Blade Runner: reconstructing films with artificial neural networks

#20

Post by astrange »

> The model also struggles to make a recognisable reconstruction when the scene is very low contrast, especially with faces.

It could be getting this wrong if his error function is calculating linear data from the given image pixels, which are in the totally not linear sRGB colorspace. That would make it badly underestimate any error in a dark image.

Quick check of the PIL docs doesn't mention gamma compensation, so they probably forgot about it. People usually do.

Post reply on HN