Shoutout to https://banana.dev , couldn't have made this demo without their hosting
Img2Prompt – Get prompts from stable diffusion generated images
31–40 of 78 posts
Re: Img2Prompt – Get prompts from stable diffusion generated images
#32this is neat, do you have any docs/posts about it? I presume it isn't on github?
Re: Img2Prompt – Get prompts from stable diffusion generated images
#33Isn’t it just CLIP ? The model that made these image generation models possible. It’s good to describe a picture but it’s not reverse engineering. The predicted prompt usually has very little in common with the actual prompt. And it’s worse when you use embeddings or fine tuned models.
What's interesting to me is that it even tries to predict the prompt on images that came straight from Stable Diffusion with no editing - which is weird because such images actually do have the prompt embedded inside of them already. (At least, that's the case for me - the prompt and parameters are stored in a tEXt chunk in the PNG file, which can be read with, for example, "pngcheck -t".)
This is something specific to the automatic 1111 version. It's just a setting, but I believe it's on by default.
Re: Img2Prompt – Get prompts from stable diffusion generated images
#34this is neat, do you have any docs/posts about it? I presume it isn't on github?
Not yet, looking into creating a write up on how it works and possibly open-sourcing
Re: Img2Prompt – Get prompts from stable diffusion generated images
#35I tried it with a cat image, and it didn't quite capture the feel:
Re: Img2Prompt – Get prompts from stable diffusion generated images
#36Earlier quoted context omitted.
This sounds a lot like the results you get from CLIP interrogation. Maybe they just use that and made another online service for it?
It is actually based on a different approach, it uses an image-captioning model finetuned on image-prompt pairs
CLIP Interrogator uses BLIP, an image captioning model, as well as trying a bunch of prompts with CLIP. I guess you mean that this model uses the captioning model to generate the complete prompt? Is the code for this one available?
Re: Img2Prompt – Get prompts from stable diffusion generated images
#37Can we also reverse engineer text? Would love text2prompt, super helpful to get better at prompting.
Re: Img2Prompt – Get prompts from stable diffusion generated images
#38Hugged to death already?
I often use terms like "sexy", "risque" etc. in the process of getting images that are quite sensible (like military people playing chess). I use img2img repeatedly looking for particular photo-film aesthetics and tend to accumulate prompts. Anyway, this would open me to charges of sexism (or worse "misogyny"), and makes me uneasy about using SD.
Edit/OH: it generates prompts for like Excel screenshots but not for images made with the img2img model at hugginface. Fascinating.
Re: Img2Prompt – Get prompts from stable diffusion generated images
#39Yeah I wouldn't say it's very close:
Tried it on last image I generated on: https://blazordiffusion.com/artifacts/50/50418_studio-ghibli...
Original Prompt:
> Studio ghibli, rocket explosion, jungle, solar, green technology, optimist future
> 8k, Bokeh effect, Cinematic Lighting, Octane Render, Iridescence, Vibrant
> by Beeple, Asher Brown Durand, Dan Mumford, Greg Rutkowski, WLOP
Img2Prompt:
> a vehicle in the grass, colorful light dust, cinematic lighting, trending on artstation, ultra detailed, art by akihito yoshida
Looks like a decent image classifier, but not useful for extracting the original stable diffusion prompt.
Re: Img2Prompt – Get prompts from stable diffusion generated images
#40Hugged to death already? I often use terms like "sexy", "risque" etc. in the process of getting images that are quite sensible (like military people playing chess). I use img2img repeatedly looking for particular photo-film aesthetics and tend to accumulate prompts. Anyway, this would open me to charges of sexism (or worse "misogyny"), and makes me uneasy about using SD. Edit/OH: it generates prompts for like Excel s…
Why does it choke on img2img creations? This is just fascinating. It gives a plausible prompt to at least a handful of real non-AI photos from DSLRs and iphones alike.