Live data from Hacker News

Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

youtube.com

41–50 of 68 posts

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#41
post #34

Worth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/92401333…

That is true, but if you don't look at them as a machine learning method, but rather as a computer graphics method, then it is quite impressive. It has the added benefit of being allowed to overfit as long as the average human does not find out. If you optimize for psychovisual metrics, GANs are fine.

Actually, GANs reach state of the art in anomaly/outlier detection and drug/molecule prediction, so there is certainly more to it than just artistic applications:

https://openreview.net/forum?id=S1EfylZ0Z

https://www.ncbi.nlm.nih.gov/pubmed/28703000

http://pubs.acs.org/doi/abs/10.1021/acs.molpharmaceut.7b0034...

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#42
post #35
post #33

Earlier quoted context omitted.

Where I live that thing alone is $17,000.

P100 is essentially 1080Ti, so you can grab 1080 Ti and have the same speed. V100 might be the expensive version, up to 10x faster.

You should look at GPU memory bandwidth for a proxy for performance when training DNNs. The P100 is about 40% faster than a 1080Ti. The V100 is only about 75% faster than a 1080Ti.

Based on this, i expect these commodity GPU servers (with 10 1080Ti cards) that cost 1/10th of the DGX-1 will be huge: https://www.servethehome.com/deeplearning11-10x-nvidia-gtx-1...

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#43

Worth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/92401333…

Seems like you could validate using random points in the latent space, and crudely verifying that the nearest neighbors in image space aren't similar.

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#44

Interesting. In another 300-500 years, I am pretty sure we will start simulating sensory experience and ultimately the past. I am not sure if I am a toy simulation of the past from the future, right now.

> In another 300-500 years

instead of "In the future", triggers me. Do you know something we do not? I am resigned to the notion that everything is up for grabs.

> I am not sure if I am a toy simulation

We need "reality discriminants". The trippy thing is if they could exist and their output is not necessarily boolean. There would be a threshold point at which beings can exist along the simulated-real spectrum, were by they can understand the output of the "reality discriminants", yet they are not real.

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#45

Worth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/92401333…

This is also relevant to your comment: https://arxiv.org/abs/1705.07663

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#46

Worth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/92401333…

[deleted]

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#47

Worth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/92401333…

My interpretation of this "problem" is that it's a radically efficient solution to the problem of generating content in absence of outside reinforcement.

Said another way: If we gave a human the task of: [make a painting of a bridge] using a handful of examples of bridges as inspiration, and they did a 1-1 copy of one of them, it would be the most efficient result. However there is generally a culturally implied task of [the new painting should not be a direct replication of one of the examples].

So this "problem" with GAN's is a novelty requirement which is not explicitly built in to the generation chain.

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#48
post #3

I’ve seen this going around twitter and the video looks cool but I don’t really understand what’s going on. Can someone explain it in layman’s terms?

Typically in a neural network, you train a single network against a single loss function that is known in advance. For example, an autoencoder is (usually) a neural network that has a chokepoint somewhere, and is trained to reconstruct the input image. Since there is a chokepoint (a layer that is significantly smaller than the input), it learns to compress the input and reconstruct it. Sort of like a lossy image compression. To train it, and tell how well it does, we can just measure the output against the input (difference between then reconstructed image and original input image). This tells us how well the network does, and gives us a well known loss function we can use in advance.

But what if we don't have a loss function? Or we don't know it? (for example, how do we even measure "what makes a face a celebrity-like face?") In that case we can train it against another network that is itself trained to differentiate between a "real" input and a "fake" input. The new network takes an image as input, and outputs a probability that the input is real or fake. We don't know the loss function, but by alternating which batch of images this network gets (fake or real), we can tell how well it does (it should estimate the real oens are real, and the fake ones fake). By training these two networks in tandem, we can use the information from the new network (the discriminator) to tell the old network (the generator) how to generate new, better images. This way we don't really need to know the loss function in advance, between the discriminator serves as our loss function.

That is the general idea. In practice, it's fairly non-trivial to get these two networks to work together nicely... often one will get much better than the other, which prevents the other from learning.

In this particular paper, they are using a technique to expand the size of the images to much larger than you would normally be able to.

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#49
post #13

Wonder how long until this is co-opted by the porn industry and what the law will have to say about it. Is it illegal to own a digital brain that can think up illegal porn?

I would say that it probably is. If you trained a network using illegal data (e.g. cp images) then not only did you have to have that data once, which is of course illegal, but the data itself is at least partially encoded in the network weights which I think should make it illegal.

I guess you might be able to get around this if you train only on legal content and can interpolate into content that would be illegal if a real recording.

However, I'm not sure whether there are any other applications for this specific interpolation scenario that would lead to it being developed, as the effort required to make it work is likely much higher.

Re: Progressive Growing of GANs for Improved Quality, Stability, Variation [video]

#50

Worth pointing out the main criticism of GANs, which is that right now researchers don't really have a way to tell if a GAN is just copying and pasting the training data or not (there is no "test set" unlike in supervised learning). And in fact an ideal GAN could just learn to output the training set. One example someone found in the generated images for this model: https://twitter.com/nalkalchbrenner/status/92401333…

An GAN that just made verbatim copies from the training set would not be able to smoothly interpolate in latent space. Also, see Fig. 10 in the paper at http://research.nvidia.com/publication/2017-10_Progressive-G....
Post reply on HN