StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery • Hacker News Board

Re: StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

#11

Post by minimaxir » Sun, Apr 04, 2021, 6:00 PM UTC

The Colab notebooks are good ways to test this out. The optimization one can render a frame at each optimization step and render as a video, which can make for some fun interpolation: https://twitter.com/minimaxir/status/1377480997684453378

Demo of global directions: https://twitter.com/minimaxir/status/1378766961937555457

Re: StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

#12

Post by mrkramer » Sun, Apr 04, 2021, 6:36 PM UTC

"Adobe Research" This will probably get added to Adobe PhotoShop.

Re: StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

#13

Post by gwern » Sun, Apr 04, 2021, 11:29 PM UTC

jhvkjhk wrote: post #2

I doubt whether this kind of AI can fully understand human languages. If the answer is no, will we create a new genre of languages serving them specifically? Imagine in the near future, programmers are not eliminated by AI, instead they code with a language looks like spoken language, but it is unnatural for human, designed for AI like this.

StyleCLIP doesn't necessarily show off what CLIP is capable of, much less what NNs right now are already capable. If you're interested, you should read through https://openai.com/blog/dall-e/ https://openai.com/blog/clip/

Re: StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

#14

Post by jacobajit » Mon, Apr 05, 2021, 12:18 PM UTC

Related, another paper using CLIP with StyleGAN for text-based semantic image manipulation from just a few days prior ("Paint by Word", where the user can select the area to be transformed): https://arxiv.org/pdf/2103.10951.pdf

Re: StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery

#15

Post by dannyw » Mon, Apr 05, 2021, 5:58 PM UTC

How much manual work is required in training the text model? Do I need to give examples?