Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

github.com

Post reply on HN
Print view View original

61–70 of 72 posts

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#61

Post by johndough »

Samin100 wrote: post #32
tgtweak wrote: post #25

Earlier quoted context omitted.

Great work on Lexica.art, it's been indispensable for finding good prompts and combinations. How much does it cost to host it? Feels like hosting 500GB of images and serving them can't be cheap.

Last month Lexica served a little over 1 billion images and the Cloudflare bill (I'm using R2 + workers) was a little over $5k. I've since gotten it down to a more reasonable amount after spending some time to re-encode the images to reduce our bandwidth usage significantly. If Lexica were running on AWS/S3 I imagine our first month's bill would be closer to $100k rather than $5k. This is only image serving, so not i…

What is the bandwidth like? I guess something in the order of a few 100 TB? Perhaps you can host this from a an "unmetered" server for $50 per month. Not sure how high your peak loads are though.

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#62

Post by avereveard »

Samin100 wrote: post #32
tgtweak wrote: post #25

Earlier quoted context omitted.

Great work on Lexica.art, it's been indispensable for finding good prompts and combinations. How much does it cost to host it? Feels like hosting 500GB of images and serving them can't be cheap.

Last month Lexica served a little over 1 billion images and the Cloudflare bill (I'm using R2 + workers) was a little over $5k. I've since gotten it down to a more reasonable amount after spending some time to re-encode the images to reduce our bandwidth usage significantly. If Lexica were running on AWS/S3 I imagine our first month's bill would be closer to $100k rather than $5k. This is only image serving, so not i…

Weird 1billion 50k request ends up about 5k on the cloudfront pricing calculator. Do you have a bandwidth estimate? Idk how you got that 100k quote

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#63

Post by nextaccountic »

vipermu wrote: post #40
nextaccountic wrote: post #29

Hey, was the specific Stable Diffusion version used to generate each image recorded anywhere in the dataset? In krea.ai, it doesn't say which version of the model was used to generate each image It appears that later versions are better in generating faces or something. Like, Stable Diffusion 1.5 vs 1.4 (I'm not sure but there's a great variability nonetheless and I wanted to know if the version of the model accounte…

Good question. All the generations in the dataset were generated with version 1.3.

Could you somehow indicate this in the web ui of krea.ai? Maybe when there are more versions

(Why aren't 1.4 images part of the dataset? Someone said they are public too)

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#64

Post by dvrp »

nextaccountic wrote: post #63
vipermu wrote: post #40

Earlier quoted context omitted.

Good question. All the generations in the dataset were generated with version 1.3.

Could you somehow indicate this in the web ui of krea.ai? Maybe when there are more versions (Why aren't 1.4 images part of the dataset? Someone said they are public too)

We’re in the process of adding them

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#65

Post by maaaaattttt »

Is there any effort being done in rating prompts in regard to the image the model output and/or what the user chose as being a satisfactory image?

I could (probably naively) imagine that this would be the next step in making these models even more pleasing to humans. Or at least in creating a GPT-based "companion" model that would suggest, from an initial subpar prompt, a prompt yielding better results.

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#66

Post by capableweb »

RandomBK wrote: post #53
indigodaddy wrote: post #42

Earlier quoted context omitted.

Why not go with a server or two or some VMs on Hetzner/Kimsufi/OVH/Netcup/BuyVM etc where they have very generous included transfer or even unmetered (BuyVM) ? I get it that everyone wants to use the trendy newest tech (workers etc or whatever the latest is), but your bill could easily be 20% (or less) of the $5k kind of numbers you are mentioning. I guess if those kind of numbers are just water under the bridge for…

Having used many different providers (though not all of the ones on your list), be very careful with suspiciously generous or unmetered anything. More often than not, you'll hit a soft limit where your performance will degrade and/or you'll get kicked for ToS violations. Most apps never hit those limits, but once you get to multi-thousands/mo in cloud bills, there's benefits in pay-as-you-go billing where the provide…

> More often than not, you'll hit a soft limit where your performance will degrade and/or you'll get kicked for ToS violations.

Hetzner does have truly unmetered root servers, been using it myself for years. If you're doing Tor, Torrents or commercial CDN traffic, they might kick you out, but for other "normal" things, they seem to have no issues with handling it as they say they will.

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#67

Post by dvrp »

edreismd wrote: post #59
vipermu wrote: post #5

Earlier quoted context omitted.

The source (Stability AI Discord) is the same, but I don't know how Sharif gathered his data.

How to access the SD dataset?

In the repository there’s a link or a `wget` statement that you can execute to download the 10M dataset.

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#68

Post by dvrp »

mod wrote: post #54

Wish the back button could find my spot on the page. Fun to explore the prompts getting results similar to what I want. Great project.

I know! I’m sorry that it doesn’t work that way.

The site is using Svelte + SvelteKit and I couldn’t find amazing Masonry components (like Masonic from React) that allow me to save and restore the scroll position easily. I can do it using hacky ways but there’s more important things to do.

I’m also still trying to figure out why Back Forward Cache is not working right away with my current implementation. I would make the site snappier and also address the issue you’re bringing.

Perhaps open-sourcing the code and figuring it out all together is the way…

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#69

Post by dvrp »

maaaaattttt wrote: post #65

Is there any effort being done in rating prompts in regard to the image the model output and/or what the user chose as being a satisfactory image? I could (probably naively) imagine that this would be the next step in making these models even more pleasing to humans. Or at least in creating a GPT-based "companion" model that would suggest, from an initial subpar prompt, a prompt yielding better results.

Well, we have a like + collection system at krea.ai.

We may use the data from there to train custom models. Kind of the same that MidJourney has, where they ask people to rate images in exchange for GPU hours as prizes.

We haven’t thought deeply about it yet.

Re: Show HN: Open Prompts – dataset of 10M Stable Diffusion generations

#70

Post by dvrp »

password4321 wrote: post #38

Show HN: I made 7k images with DALL-E 2 to create a reference/inspiration table https://generrated.com https://news.ycombinator.com/item?id=32824448

I already knew your site. I don’t recall where I saw it. I shared it with Victor right away because of the sheer amount of content! Insane!

In fact, I think thought for a while about how to potentially integrate it with Krea somehow. But I came up empty. If you have any ideas, please reach out via twitter!

Post reply on HN