Also See: 151 Illusions & Visual Phenomena with explanations by Michael Bach
Illusion Diffusion: Optical Illusions Using Stable Diffusion
11–12 of 12 posts
Re: Illusion Diffusion: Optical Illusions Using Stable Diffusion
#12This is great. Those examples are not the best quality, but they're impressive. That prompted me to generate ambigrams with stable diffusion. The results looked odd, as ambigrams tend to, but the "text" was largely illegible. I wonder when the state of the art will be able to handle that request.
It's odd that image AIs are not ready to overlay text. If you ask Dall-E or Midjourney also to say a few letters they do lots of nearest random neighbors by not just scrambling the idea of the word but also scribbling anything on top that it thinks looks remotely like writing but is not in any language. Maybe it's still developing the ability to read and maybe secretly creating a completely new script and lang.
It's a side effect of the way the text input is represented before being used by the model. It doesn't get the text as a sequence of chars but as a sequence of tokens.
This paper [1] shows that giving character-level awareness to the model can improve the "visual spelling".