I wonder if you could make the noise inputs work again by using the same process as for the latent code - generate the noise in the frequency domain, and apply the same shift and careful downsampling. If you apply the same shift to the noise as to the latent code, then maybe the whole thing will still be equivariant? In other words, it seems like the problem with the per-pixel noise inputs is that they stay stationary while the latent is shifted, so just shift them also!
Alias-Free GAN
11–20 of 83 posts
Re: Alias-Free GAN
#12This group of researchers consistently demonstrates a degree of empirical rigor that is unmatched across any other ML lab in industry or academia - remarkable empirical results as always, reproducible experiments, open-source and well-engineered codebase, and valuable insights about low-level learning dynamics and high-level emergent artifacts. Applied ML wouldn't have such a bad rap if more researchers held themselves to similar standards.
Re: Alias-Free GAN
#13After styleGAN-2 came out, I couldn't image what improvements could be made over it. This work is truly impressive. The comparisons are illuminative: StyleGAN2's mapping of texture to specific pixel location looks very similar to poorly implemented video-game textures. Perhaps future GAN improvements could come from tricks used in non-AI graphic development.
>I couldn't image what improvements could be made over it Still has the telltale of mismatched ears and/or earrings. This seems the most reliable way to recognize them. Well, and the nondescript background.
Mismatched reflections across eyes is the dead give-away for me.
Re: Alias-Free GAN
#14That's starting to be high enough quality that you could start considering using that for some Hollywood-grade special effects. That beach morph stuff is pretty impressive. Faces, perhaps not quite there yet because we are so hyper-focused on those biologically, but you could make one heck of a drug trip scene or a Doctor Strange-esque scene with much less effort with some of those techniques, effort perhaps even getting down to the range of Youtuber videos in the near enough future.
Re: Alias-Free GAN
#15I expect this work will feed back into removing the aliasing artifacts you sometimes get when using DLSS in games.
Re: Alias-Free GAN
#16After styleGAN-2 came out, I couldn't image what improvements could be made over it. This work is truly impressive. The comparisons are illuminative: StyleGAN2's mapping of texture to specific pixel location looks very similar to poorly implemented video-game textures. Perhaps future GAN improvements could come from tricks used in non-AI graphic development.
>I couldn't image what improvements could be made over it Still has the telltale of mismatched ears and/or earrings. This seems the most reliable way to recognize them. Well, and the nondescript background.
Teeth too. Partially covered objects in 3D space have been hard for a GAN to figure out. (See also hands)
I wonder what dataset you could even use to tell a GAN about human internals. 3D renders of a skull with various layers removed?
Re: Alias-Free GAN
#17The previous approaches learned screen-space-textures for different features and a feature mask to compose them.
Now it seems to actually learn the topology lines of the human face [0], as 3D artists would learn them [1] when they study anatomy. It also uses quad grids and even places the edge loops and poles in similar places.
[0] https://nvlabs-fi-cdn.nvidia.com/_web/alias-free-gan/img/ali... [1] https://i.pinimg.com/originals/6b/9a/0c/6b9a0c2d108b2be75bf7...
Re: Alias-Free GAN
#18That beach interpolation is begging for a music video
Re: Alias-Free GAN
#19Great work!
Re: Alias-Free GAN
#20The first two demo videos are interesting examples of using StyleCLIP's global directions to guide an image toward a "smiling face" as noted in that paper with smooth interpolation: https://github.com/orpatashnik/StyleCLIP I had ran a few chaotic experiments with StyleCLIP a few months ago which would work very well with smooth interpolation: https://minimaxir.com/2021/04/styleclip/
That first picture of mark zuckerberg smiling is just straight up cursed. Interesting write up though.