Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

github.com

Post reply on HN
Print view View original

1–10 of 72 posts

Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#1

Post by vipermu »

Open Prompts is the dataset used to build krea.ai. The data comes from the Stability AI Discord and includes around 10M images from 2M prompts. You can use it for creating semantic search engines of prompts, training LLMs, fine-tuning image-to-text models like BLIP, or extracting insights from the data—like the most common combinations of modifiers.

Show HN: Open Prompts – dataset of 10M Stable Diffusion generations
github.com

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#3

Post by vipermu »

Philomath wrote: post #2

That's amazing, thanks for sharing. For how long have you been gathering this data?

We do not have a continuous system, the data is a mix between our own crawled generations and the dataset published by Dave Caruso (https://github.com/paperdave). With our crawler, we were able to get about 100k generations per day.

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#6

Post by dvrp »

cercatrova wrote: post #4

How does it compare with https://lexica.art ?

Krea dev here.

Lexica is a search engine (like krea.ai), but it doesn’t allow you to create collections or like generations.

Regarding the API, both have public APIs although I’m not sure if you can paginate through several search results using the public Lexica API. In the Krea Prompts API, you can do cursor-based pagination.

Finally, Lexica API allows you to do CLIP-based search, but with Krea we are using PostgreSQL full-text search (for now). However, the code to do CLIP search with the dataset (including reverse image search) is in the repository.

(edit: also, nor Lexica nor other search engines or similar products are offering the dataset afaik.)

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#9

Post by dvrp »

Oras wrote: post #8

This is fantastic. A few days ago I was checking PromptBase [0] and thought it was a really good idea. Yours just took it to the next level being free with massive amount of data. Great work. [0] https://promptbase.com/

Thanks!

I’d love to integrate our crawler with GitHub Actions and make it a self-updating dataset…

There’s so much stuff to do!

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#10

Post by dvrp »

rhacker wrote: post #7

someone should do a thing that lets strangers critique AI by allowing them to select: A person has a glitchy face in this photo. A person has a glitchy body in this photo. etc.. and then train the AI to have a fixup pass.

Interesting. Kind of like Scale AI for generative AI.
Post reply on HN