This Marketing Blog Does Not Exist • Hacker News Board

Re: This Marketing Blog Does Not Exist

#51

Post by gwern » Thu, Jun 20, 2019, 2:52 AM UTC

gwern wrote: post #44

TeMPOraL wrote: post #18

Earlier quoted context omitted.

It's going in the wrong direction. Sure, it can conjure up a random picture of a face. But for purposes or manga/anime building, it's near useless, as it can't generate that face in multiple perspectives, or with different kinds of facial expressions. This Waifu truly does not exist - neither in prior publications, nor in the algorithm itself.

That's incorrect. You can generate the same face with multiple facial expressions by either doing style transfer onto a single face from other faces using the style noise (as demonstrated in the StyleGAN paper & video, and you can see further examples with my anime StyleGAN faces), or you can directly control a face by modifying its latent vector as desired, once you've reverse-engineered the latent vector (as can be…

For a recent cool example, http://everyoneishappy.com/portfolio/waifu-synthesis-real-ti... controls the mouth for sing-along syncing.

Re: This Marketing Blog Does Not Exist

#52

Post by TeMPOraL » Thu, Jun 20, 2019, 7:53 AM UTC

gwern wrote: post #44

TeMPOraL wrote: post #18

Earlier quoted context omitted.

It's going in the wrong direction. Sure, it can conjure up a random picture of a face. But for purposes or manga/anime building, it's near useless, as it can't generate that face in multiple perspectives, or with different kinds of facial expressions. This Waifu truly does not exist - neither in prior publications, nor in the algorithm itself.

That's incorrect. You can generate the same face with multiple facial expressions by either doing style transfer onto a single face from other faces using the style noise (as demonstrated in the StyleGAN paper & video, and you can see further examples with my anime StyleGAN faces), or you can directly control a face by modifying its latent vector as desired, once you've reverse-engineered the latent vector (as can be…

Wow. Thanks for that; I'm very happy to have been wrong about this issue, and I've just learned a lot from reading your Faces page.

One issue I notice in your video of interactive face editing is what I'd call "crosstalk" (there's probably a more formal term for this) - you alter a specific slider, and beyond the targeted modification, the entire face, shirt and background changes slightly. Is there any work on eliminating such crosstalk, so one could e.g. stitch together a believable video from such dynamically modified face?

Re: This Marketing Blog Does Not Exist

#53

Post by gwern » Thu, Jun 20, 2019, 11:34 AM UTC

TeMPOraL wrote: post #52

gwern wrote: post #44

Earlier quoted context omitted.

That's incorrect. You can generate the same face with multiple facial expressions by either doing style transfer onto a single face from other faces using the style noise (as demonstrated in the StyleGAN paper & video, and you can see further examples with my anime StyleGAN faces), or you can directly control a face by modifying its latent vector as desired, once you've reverse-engineered the latent vector (as can be…

Wow. Thanks for that; I'm very happy to have been wrong about this issue, and I've just learned a lot from reading your Faces page. One issue I notice in your video of interactive face editing is what I'd call "crosstalk" (there's probably a more formal term for this) - you alter a specific slider, and beyond the targeted modification, the entire face, shirt and background changes slightly. Is there any work on elimi…

My theory is that the latent space isn't perfectly 'disentangled' (there's not really anything in StyleGAN akin to InfoGAN which would force it to be exactly linear/disentangled, it just nicely works out roughly disentangled), so when you train a linear model on the tags+latent and manipulate it linearly, you get slight changes in correlated features. Thus, the slight 'glitchiness'. Not something you'd notice in image samples, but more apparent when you generate interpolations of the 'same' face.

If you mess with the style noise only and leave the latent vector alone, the glitchiness should be less, but you also lose a lot of possible control. Training a more powerful nonlinear classifier like a random forests should be able to compensate for the remaining nonlinearity in the latents, I think, but then it's harder to do the actual modification of the latent vector, of course, since you're no longer simply tweaking one variable but many simultaneously to try to hold the other features constant. One could try to brute force it by doing blackbox optimization of the nonlinear classifier... but I haven't seen anyone try that.