SuperPrompt: Better Text to Image Prompts in 77M Parameters
brianfitzgerald.xyz
SuperPrompt: Better Text to Image Prompts in 77M Parameters
1–10 of 33 posts
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#2Nice. I've been using GPT-4-turbo with a custom system prompt for this until now. Going to try this out.
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#3I'm surprised this isn't getting more love. I love the concept of finetuned, hyper-specific, tiny LLMs. Of course, the data is the most important part.
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#4This is neat and some thing (aka text "expanders") that I imagine a lot of the commercial offerings (midjourney, etc) are using behind the scenes.
This seems to be targeting SDXL workflows, but in my experience a lot of the custom checkpoints derived from SDXL can have widely divergent recommended prompting styles ranging from natural language to just a list of booru tags.
So I'm guessing this is really only optimized for base SDXL, but I would be curious to see how well it worked on some of the more SOTA SDXL checkpoints such as juggernaut and unstable.
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#5It’s impressive how well the T5 family of models has aged, even compared to newer LLM architectures.
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#6Great work! I'd recommend including the "max_length=77" parameter in your example, and it seems like the huggingface hosted interface is broken because of the tokenizer. Also, I think your website link on X is outdated.
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#7Great work! I'd recommend including the "max_length=77" parameter in your example, and it seems like the huggingface hosted interface is broken because of the tokenizer. Also, I think your website link on X is outdated.
will fix these, thanks for the heads up!
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#8Awesome work! I’d love to see how this could be integrated with existing tools like InvokeAI.
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#9This is neat and some thing (aka text "expanders") that I imagine a lot of the commercial offerings (midjourney, etc) are using behind the scenes. This seems to be targeting SDXL workflows, but in my experience a lot of the custom checkpoints derived from SDXL can have widely divergent recommended prompting styles ranging from natural language to just a list of booru tags. So I'm guessing this is really only optimize…
I haven't tested extensively with non SDXL based checkpoints but there's nothing really SDXL specific about the model; if you're using a fine-tune that's trained on booru-style tags, it will probably not work as well - but otherwise it should work just fine. And in that case, just fork the project and tune it on however your fine-tune prompts best :)
Re: SuperPrompt: Better Text to Image Prompts in 77M Parameters
#10Awesome work! I’d love to see how this could be integrated with existing tools like InvokeAI.
As Invoke is open-source and already has transformers as a dependency, it should be pretty easy to add.