Some notes on the Stable Diffusion safety filter • Hacker News Board

Re: Some notes on the Stable Diffusion safety filter

#21

Post by formerly_proven » Fri, Nov 18, 2022, 6:08 PM UTC

tifik wrote: post #4

> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content without consent of the people who might see it I understand that it's their TOS and they can put pretty much anything in there, but this item seems... odd. I don't really know why exactly this stands out to me. Maybe it's because it's practically un-enforceable? Are…

Stable Diffusion is developed at LMU Munich and this particular line basically paraphrases § 184 of the German criminal code, which makes it a misdemeanor crime to put porn in places reachable by minors or to show porn to someone without being asked to do so, among other things. I dunno why they felt compelled to include it though.

Regarding your examples, most of these are technically criminal in Germany, because the only legally safe way to have a place not-reachable-by-minors means adhering to German youth protection laws, which you're not going to, just like every porn site, Twitter, Reddit etc.

Re: Some notes on the Stable Diffusion safety filter

#22

Post by criddell » Fri, Nov 18, 2022, 6:10 PM UTC

minimaxir wrote: post #2

Unfortunately the safety filters have enough false positives (basically any image with a large amount of fleshy color) to the point that it's just easier to disable it and handle it manually.

And then the CSAM filter on your device reports you to some authority.

Re: Some notes on the Stable Diffusion safety filter

#23

Post by BoorishBears » Fri, Nov 18, 2022, 6:13 PM UTC

criddell wrote: post #22

minimaxir wrote: post #2

Unfortunately the safety filters have enough false positives (basically any image with a large amount of fleshy color) to the point that it's just easier to disable it and handle it manually.

And then the CSAM filter on your device reports you to some authority.

If you have things on your device that match entries in the CSAM database, yes there's a chance you're a victim of a targeted attack taking advantage of highly experimental collisions... but the odds you "accidentally generated" that content are not realistic.

Re: Some notes on the Stable Diffusion safety filter

#24

Post by sillysaurusx » Fri, Nov 18, 2022, 6:17 PM UTC

Animats wrote: post #9

netruk44 wrote: post #3

Earlier quoted context omitted.

That'll only work for a little while longer (for future named big-public-release models, obviously the cat's out of the bag for the current version of stable diffusion), right up until the point where they incorporate the filter into the training process. At which point, the end model users get to download will be incapable of producing anything that comes close to triggering the filter, and there will be no way to w…

That problem is being solved. Pornhub now has an AI R&D unit.[1] Their current project is to upscale and colorize out of copyright vintage porn. As a training set, they use modern porn. They point out that they have access to a big training set. Next step, porn generation. [1] https://www.pornhub.com/art/remastured

For a glimpse at what’s possible:

https://www.reddit.com/r/unstablediffusion

https://www.reddit.com/r/aiwaifu

I’ve been trying to generate tentacle porn since 2019 or so. It’s the whole reason I got into AI. We’re finally there, and it only took three years.

Can’t wait to see what 2026 brings. http://n.actionsack.com/pic/media%2FFh08F_hXkAAhalt.jpg

Re: Some notes on the Stable Diffusion safety filter

#25

Post by langitbiru » Fri, Nov 18, 2022, 6:24 PM UTC

TaylorAlexander wrote: post #19

netruk44 wrote: post #3

Earlier quoted context omitted.

That'll only work for a little while longer (for future named big-public-release models, obviously the cat's out of the bag for the current version of stable diffusion), right up until the point where they incorporate the filter into the training process. At which point, the end model users get to download will be incapable of producing anything that comes close to triggering the filter, and there will be no way to w…

This prediction doesn’t track with what is already happening. Dreambooth is allowing all kinds of people to fine tune their own models at home with nvidia graphics cards, and people are sharing all kinds of updated models that do really well at specific art styles or with NSFW subjects. Go check the nsfw subreddit unstable_diffusion for examples. It seems lots of people are training nsfw models with their own preferr…

The AI-generated art with Dreambooth works only for avatar type pics. It cannot create fancy gestures (doing a complicated movement with hands, like patting a cat). For now.

Re: Some notes on the Stable Diffusion safety filter

#26

Post by sacrosancty » Fri, Nov 18, 2022, 6:39 PM UTC

No post body was provided.

Re: Some notes on the Stable Diffusion safety filter

#27

Post by yeet_yeet_yeet » Fri, Nov 18, 2022, 6:43 PM UTC

BoorishBears wrote: post #23

criddell wrote: post #22

Earlier quoted context omitted.

And then the CSAM filter on your device reports you to some authority.

If you have things on your device that match entries in the CSAM database, yes there's a chance you're a victim of a targeted attack taking advantage of highly experimental collisions... but the odds you "accidentally generated" that content are not realistic.

>not realistic

The odds are zero.

1/2^256 = 0.

In cryptography these odds are treated as zero until you generate close to 2^128 images.

Unfortunately there's no word in natural English to describe how unlikely. The most precise is "zero".

Re: Some notes on the Stable Diffusion safety filter

#28

Post by [deleted] » Fri, Nov 18, 2022, 6:45 PM UTC

Wistar wrote: post #10

langitbiru wrote: post #5

Earlier quoted context omitted.

"Gal Gadot wearing green suit" triggered it while "Tom Cruise wearing green suit" didn't.

As did "young children watching sunset," but not "young boy and girl watching sunset."

[deleted]

Re: Some notes on the Stable Diffusion safety filter

#29

Post by properparity » Fri, Nov 18, 2022, 7:16 PM UTC

tifik wrote: post #4

> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content without consent of the people who might see it I understand that it's their TOS and they can put pretty much anything in there, but this item seems... odd. I don't really know why exactly this stands out to me. Maybe it's because it's practically un-enforceable? Are…

We need to let it completely loose and get everyone exposed to it everywhere so that maybe we can finally get rid of this insane taboo and uptightness about sex and nudity we have in society.

Re: Some notes on the Stable Diffusion safety filter

#30

Post by octagons » Fri, Nov 18, 2022, 7:33 PM UTC

properparity wrote: post #29

tifik wrote: post #4

> Using the model to generate content that is cruel to individuals is a misuse of this model. This includes, but is not limited to: ... >+ Sexual content without consent of the people who might see it I understand that it's their TOS and they can put pretty much anything in there, but this item seems... odd. I don't really know why exactly this stands out to me. Maybe it's because it's practically un-enforceable? Are…

We need to let it completely loose and get everyone exposed to it everywhere so that maybe we can finally get rid of this insane taboo and uptightness about sex and nudity we have in society.

Nudity? Yes. Pornography? No.