Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Art Generators Can Be Fooled into Making NSFW Images

spectrum.ieee.org

Post reply on HN
Print view View original

61–70 of 193 posts

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#61

Post by wruza »

teaearlgraycold wrote: post #42
pdntspa wrote: post #36

Earlier quoted context omitted.

And so, once again, everybody else suffers because "think of the children"

[flagged]

We don’t actually have numbers on how many people ended their life for being unable to generate porn they want.

Before calling this idea absurd or too far fetched (which it likely is), think about those who ought to end their life because of pics they don’t want and compare the fetch distances.

It’s amusing how society tends to ignore a thousand miles leading to an unfortunate event and only focuses on the last one. This way of thinking leaves everyone suffering right on the edge without ever addressing a whole mountain of issues on which that edge situates.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#62

Post by Mountain_Skies »

simion314 wrote: post #31
kelseyfrog wrote: post #7

I always assumed that image generation services had prompt and output filtering. The reason I believe this is that I've crafted prompts that seemed to pass the prompt filter, generated, and then were unable to be displayed[1]. Presumably there is an output filter that determines if the generation violates ToS against things like nudity. Though in my case, I wasn't able to generate images that too closely resembled An…

I had similar experience with OpenAI GPT API, following their rules we check the customer prompts with their endpoint for "safety" , then if it is safe we continue and generate the content. But sometimes even if the prompt was "safe" the result would be blocked as unsafe, because the AI either generated something "unsafe" from a safe prompt or the output filter had a false positive. Of course we had to pay for someth…

Beige color clothes are to be avoided if any of the people in the image are young adults or younger. The input prompt filter correctly understands that the user isn't trying to generate anything "unsafe", but the output filter sees clothes that are too close to Caucasian flesh tone on someone younger than middle age and flags the output. Leave everything as-is but change the clothing color to something darker or leave the color alone and exclude children and young adults, then things come out fine. Though I can't see the flagged images, it seems clear that they're not generated nudes but false positives.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#63

Post by judge2020 »

eternauta3k wrote: post #51
judge2020 wrote: post #30

Earlier quoted context omitted.

Edit: the author removed it.

Don't see a way to report this on Github.

There's no secrets here, just IP addresses and hostnames, so there's no real security issue. If you want to report it to the author you can try to message them or create an issue.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#64

Post by gustavus »

So I've been wondering lately it seems there is a lot of effort going into making sure people can't generate NSFW images, but I wonder if a lot of this outrage isn't from the tech side (they tend to be fairly open about that stuff) but rather if it is coming from the adult entertainment industry, as this kind of thing literally destroys their entire business model.

Who needs their adladen videos or their salacious acting when one can generate an exact description of exactly what they want?

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#65

Post by Mountain_Skies »

tasubotadas wrote: post #21

Together with fancy silicon chips, AI models and everything else that's being exported from the US , the world has to endure this puritanistic views export too.

What fancy silicon chips does the United States refuse to export due to puritanical views?

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#66

Post by creer »

Spivak wrote: post #52
egeozcan wrote: post #29

Earlier quoted context omitted.

Let's focus on trying to raise a mentally healthier generation and take mental illness more seriously, rather than banning / getting rid of all the things that might cause someone (let alone a teenager) to commit suicide (an infinite set).

Boiling the ocean type solutions are the same as doing nothing. A better solution would be to make creating such things very illegal and bring down the same wrath we do for CSAM. There is no amount of "mental health" that can protect you against targeted bullying of that magnitude. I really wish we had a universe simulator so you could see how miserably you would fail to survive an environment like that.

Isn't bullying already illegal and why aren't you spending that effort on enforcing THAT?

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#67

Post by pdntspa »

teaearlgraycold wrote: post #42
pdntspa wrote: post #36

Earlier quoted context omitted.

And so, once again, everybody else suffers because "think of the children"

[flagged]

More of a general observation about how white knights use "the children" like a truncheon to bash on the things they don't like

And who fucking cares if it's generating porn. There's nothing wrong with sex or sexual desire. Who are you to judge? Let people have their fun.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#68

Post by joenot443 »

candiddevmike wrote: post #32
justapassenger wrote: post #22

Earlier quoted context omitted.

[flagged]

Like bullying, there's a fix for it, but schools (especially public schools) typically lack the resources, capabilities, and incentives to do anything about it.

AI stuff aside, what's the cure for bullying?

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#69

Post by pdntspa »

wruza wrote: post #61
teaearlgraycold wrote: post #42

Earlier quoted context omitted.

[flagged]

We don’t actually have numbers on how many people ended their life for being unable to generate porn they want. Before calling this idea absurd or too far fetched (which it likely is), think about those who ought to end their life because of pics they don’t want and compare the fetch distances. It’s amusing how society tends to ignore a thousand miles leading to an unfortunate event and only focuses on the last one.…

It's extra amusing (read: horrifying) how moralists use children to manipulate people and bypass their sense of reason to go straight for the monkeybrain.

As a society, we need to call this out and shut it down as soon as it arises. Preferably whilst shutting down the people and organizations who do it.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#70

Post by ipaddr »

Spivak wrote: post #52
egeozcan wrote: post #29

Earlier quoted context omitted.

Let's focus on trying to raise a mentally healthier generation and take mental illness more seriously, rather than banning / getting rid of all the things that might cause someone (let alone a teenager) to commit suicide (an infinite set).

Boiling the ocean type solutions are the same as doing nothing. A better solution would be to make creating such things very illegal and bring down the same wrath we do for CSAM. There is no amount of "mental health" that can protect you against targeted bullying of that magnitude. I really wish we had a universe simulator so you could see how miserably you would fail to survive an environment like that.

If a naked picture of you is causing you to choose suicide I think we need to fix how society views nudity.

The answer isn't to make nudity illegal. We're going to end up blinding ourselves on purpose to avoid seeing our own nudity if we go down your road.

Post reply on HN