Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Some notes on the Stable Diffusion safety filter

vickiboykis.com

Post reply on HN
Print view View original

11–20 of 88 posts

Re: Some notes on the Stable Diffusion safety filter

#11

Post by minimaxir »

netruk44 wrote: post #8
pifm_guy wrote: post #6

Earlier quoted context omitted.

Fine-tuning is pretty cheap compared to the original training run - perhaps just 1% of the cost. Totally within reach of a consortium of.... "entertainment specialists".

I know a person who fine-tuned stable diffusion, and he said it took 2 weeks of 8xA100 80 GB training time, costing him somewhere between $500-$700 (he got a pretty big discount, too, at today's prices for peer GPU rental it would be over $1,000). Sure, it's peanuts compared to what it must have cost to train stable diffusion from scratch. However, I think most normal people would not consider spending $500 to fine-t…

For spot-finetuning with Dreambooth (not as good as full-finetuning but can get a specific subject/style much faster), it can be done with about $0.08 of GPU compute, although optimizing it is harder.

https://huggingface.co/docs/diffusers/training/dreambooth

Re: Some notes on the Stable Diffusion safety filter

#12

Post by cookingrobot »

netruk44 wrote: post #8
pifm_guy wrote: post #6

Earlier quoted context omitted.

Fine-tuning is pretty cheap compared to the original training run - perhaps just 1% of the cost. Totally within reach of a consortium of.... "entertainment specialists".

I know a person who fine-tuned stable diffusion, and he said it took 2 weeks of 8xA100 80 GB training time, costing him somewhere between $500-$700 (he got a pretty big discount, too, at today's prices for peer GPU rental it would be over $1,000). Sure, it's peanuts compared to what it must have cost to train stable diffusion from scratch. However, I think most normal people would not consider spending $500 to fine-t…

You can fine tune stable diffusion for $10 using this service: https://www.strmr.com/

It works super well for putting yourself in the images, the likeness is fantastic.

It’s obviously a small training process, they only take 20 images, but it works.

Re: Some notes on the Stable Diffusion safety filter

#13

Post by netruk44 »

minimaxir wrote: post #11
netruk44 wrote: post #8

Earlier quoted context omitted.

I know a person who fine-tuned stable diffusion, and he said it took 2 weeks of 8xA100 80 GB training time, costing him somewhere between $500-$700 (he got a pretty big discount, too, at today's prices for peer GPU rental it would be over $1,000). Sure, it's peanuts compared to what it must have cost to train stable diffusion from scratch. However, I think most normal people would not consider spending $500 to fine-t…

For spot-finetuning with Dreambooth (not as good as full-finetuning but can get a specific subject/style much faster), it can be done with about $0.08 of GPU compute, although optimizing it is harder. https://huggingface.co/docs/diffusers/training/dreambooth

Are these services using textual-inversion? If so, I have to wonder how well they would work on a stable diffusion model that was trained with the filter in place from the start, so that it couldn't generate anything close to the filter.

As it is right now, stable diffusion can generate adult imagery by itself, however it seems like it's been fine-tuned after the fact to try to 'cover up' that fact as much as they could before releasing the model publicly.

Re: Some notes on the Stable Diffusion safety filter

#14

Post by jimbob45 »

netruk44 wrote: post #7
tifik wrote: post #4

> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content without consent of the people who might see it I understand that it's their TOS and they can put pretty much anything in there, but this item seems... odd. I don't really know why exactly this stands out to me. Maybe it's because it's practically un-enforceable? Are…

I think the issue they're mainly worried about might be exemplified with a prompt of 'my little pony'. A children's show with quite a lot of adult imagery associated with it on the internet. A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. Did the…

A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet.

So everyone has to have gimpy AI just because parents can't be expected to take responsibility for what their child does and does not see? Why the fuck is a child being allowed to play with something that can very easily spit out salacious images accidentally? Wouldn't it be significantly easier to add censorship to the prompt input instead? It seems like these tech companies see yet another opportunity to add censorship to their products and can hardly hide their giddy excitement.

Re: Some notes on the Stable Diffusion safety filter

#15

Post by gpderetta »

netruk44 wrote: post #13
minimaxir wrote: post #11

Earlier quoted context omitted.

For spot-finetuning with Dreambooth (not as good as full-finetuning but can get a specific subject/style much faster), it can be done with about $0.08 of GPU compute, although optimizing it is harder. https://huggingface.co/docs/diffusers/training/dreambooth

Are these services using textual-inversion? If so, I have to wonder how well they would work on a stable diffusion model that was trained with the filter in place from the start, so that it couldn't generate anything close to the filter. As it is right now, stable diffusion can generate adult imagery by itself, however it seems like it's been fine-tuned after the fact to try to 'cover up' that fact as much as they co…

As far as I understand textual inversion != Dreambooth != Actual fine-tuning

Re: Some notes on the Stable Diffusion safety filter

#16

Post by RC_ITR »

jimbob45 wrote: post #14
netruk44 wrote: post #7

Earlier quoted context omitted.

I think the issue they're mainly worried about might be exemplified with a prompt of 'my little pony'. A children's show with quite a lot of adult imagery associated with it on the internet. A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. Did the…

A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. So everyone has to have gimpy AI just because parents can't be expected to take responsibility for what their child does and does not see? Why the fuck is a child being allowed to play with somethin…

Because in aggregate, children seeing those things has impacts on society.

Like sure would it be better if parents monitored their children’s 4chan use? Ofc.

Is that at all a practical approach to eliminating Elliot Roger idolization? No.

Re: Some notes on the Stable Diffusion safety filter

#17

Post by netruk44 »

jimbob45 wrote: post #14
netruk44 wrote: post #7

Earlier quoted context omitted.

I think the issue they're mainly worried about might be exemplified with a prompt of 'my little pony'. A children's show with quite a lot of adult imagery associated with it on the internet. A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. Did the…

A child entering this prompt is probably expecting one thing, but the internet is filled with pictures of another nature. There are possibly more adult 'my little pony' images than screenshots of the show on the internet. So everyone has to have gimpy AI just because parents can't be expected to take responsibility for what their child does and does not see? Why the fuck is a child being allowed to play with somethin…

Just to be clear, the child was just an example of someone who could theoretically experience 'cruel' treatment from the current version of stable diffusion. I'm absolutely not recommending people let their children use the model unsupervised. It doesn't have to be a parenting problem, though.

The same could be said (for example) of a random mother trying to get inspiration for a 'my little pony' birthday cake for their child, and being presented with the 'other' kind of image unintentionally, without their consent. I think they would be justifiably upset in that situation.

If we were to imagine someone attempting to put stable diffusion into some future consumer product, I think they would have to be concerned about these kinds of scenarios. Therefore, the scientists are trying to figure out how to accomplish the filtering.

FWIW, I don't think a model could be made that actively prevented people from using their own NSFW training data. The only difference in the future will be that the public models won't be able to do it 'for free' with no modifications needed. You'll have to train your own model, or wait for someone else to train one.

Re: Some notes on the Stable Diffusion safety filter

#18

Post by seaal »

netruk44 wrote: post #13
minimaxir wrote: post #11

Earlier quoted context omitted.

For spot-finetuning with Dreambooth (not as good as full-finetuning but can get a specific subject/style much faster), it can be done with about $0.08 of GPU compute, although optimizing it is harder. https://huggingface.co/docs/diffusers/training/dreambooth

Are these services using textual-inversion? If so, I have to wonder how well they would work on a stable diffusion model that was trained with the filter in place from the start, so that it couldn't generate anything close to the filter. As it is right now, stable diffusion can generate adult imagery by itself, however it seems like it's been fine-tuned after the fact to try to 'cover up' that fact as much as they co…

I believe the safety filter is trivial to disable since it was added in one of the last commits prior to Stable Diffusion’s public release and not baked into the model, therefore most forks just remove the safety checker code [1]

As far as textual inversion, JoePenna’s Dreambooth [2] implementation uses Textual Inversion.

[1] https://github.com/CompVis/stable-diffusion/commit/a6e2f3b12... [2] https://github.com/JoePenna/Dreambooth-Stable-Diffusion

Re: Some notes on the Stable Diffusion safety filter

#19

Post by TaylorAlexander »

netruk44 wrote: post #3
minimaxir wrote: post #2

Unfortunately the safety filters have enough false positives (basically any image with a large amount of fleshy color) to the point that it's just easier to disable it and handle it manually.

That'll only work for a little while longer (for future named big-public-release models, obviously the cat's out of the bag for the current version of stable diffusion), right up until the point where they incorporate the filter into the training process. At which point, the end model users get to download will be incapable of producing anything that comes close to triggering the filter, and there will be no way to w…

This prediction doesn’t track with what is already happening. Dreambooth is allowing all kinds of people to fine tune their own models at home with nvidia graphics cards, and people are sharing all kinds of updated models that do really well at specific art styles or with NSFW subjects. Go check the nsfw subreddit unstable_diffusion for examples. It seems lots of people are training nsfw models with their own preferred data sets and last I saw someone merged all those checkpoints together in to one model.

So if I made a prediction it would be that the training sets for open models from big companies will get scrubbed of nsfw content and then nerds on Reddit will just release their own versions with it added in, and the big companies will make sure everyone knows they didn’t add that stuff and that’s where it will stand.

Re: Some notes on the Stable Diffusion safety filter

#20

Post by netruk44 »

TaylorAlexander wrote: post #19
netruk44 wrote: post #3

Earlier quoted context omitted.

That'll only work for a little while longer (for future named big-public-release models, obviously the cat's out of the bag for the current version of stable diffusion), right up until the point where they incorporate the filter into the training process. At which point, the end model users get to download will be incapable of producing anything that comes close to triggering the filter, and there will be no way to w…

This prediction doesn’t track with what is already happening. Dreambooth is allowing all kinds of people to fine tune their own models at home with nvidia graphics cards, and people are sharing all kinds of updated models that do really well at specific art styles or with NSFW subjects. Go check the nsfw subreddit unstable_diffusion for examples. It seems lots of people are training nsfw models with their own preferr…

I agree with your prediction. Sorry, I was unclear in my post, and left that part unsaid. I agree that it will likely just be the big newly released 'base' models that will be scrubbed of NSFW images, but there's really no way to prevent these models from making those kinds of images at all.

It will only take some dedicated individuals, which I know there is no shortage of.

Post reply on HN