Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI Art Generators Can Be Fooled into Making NSFW Images

spectrum.ieee.org

Post reply on HN
Print view View original

161–170 of 193 posts

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#161

Post by pxoe »

ipaddr wrote: post #70
Spivak wrote: post #52

Earlier quoted context omitted.

Boiling the ocean type solutions are the same as doing nothing. A better solution would be to make creating such things very illegal and bring down the same wrath we do for CSAM. There is no amount of "mental health" that can protect you against targeted bullying of that magnitude. I really wish we had a universe simulator so you could see how miserably you would fail to survive an environment like that.

If a naked picture of you is causing you to choose suicide I think we need to fix how society views nudity. The answer isn't to make nudity illegal. We're going to end up blinding ourselves on purpose to avoid seeing our own nudity if we go down your road.

and it's not making nudity illegal, it's about nonconsensual creation and distribution of images with nudity. which may be connected with bullying, blackmail, extortion, or may not be.

>If a naked picture of you is causing you to choose suicide

really great response. almost as good as 'if rape is causing you to choose suicide'. 'maybe we need to fix how society views rape'. nonconsensual nude imagery and pornography is much closer to rape than not.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#162

Post by pxoe »

Permik wrote: post #93
Spivak wrote: post #78

Earlier quoted context omitted.

It's not a naked photo. It's 1. Complete loss of privacy and control over your image. 2. Shame and embarrassment, especially in high school where every boy will look at you different and reduce you to a sexual object. 3. Public humiliation -- those photos can and do lead to ridicule and bullying. You're the one who is naked while everyone else is clothed. 4. Destroys self-esteem -- high school is already terrible in…

It's just a naked photo, you're just describing a broken (social/social hierarchy) system. Don't hate the players (bully, bullied), hate the game.

Banning non-consensual nude imagery would "fix" that "broken system". The "game" would be fixed, if those games would be made illegal. So why don't you like that as a solution? Is it supposed to be just "hated" without actually being fixed?

You're conflating 'nudity' in general, and non-consensual generated nude images, but like, these things are just different, and are treated differently. There isn't a misunderstanding that nudity in, say, art, is different from nudity in porn, or that there's difference between consensual and nonconsensual depictions, or whether something involves made-up characters and real people. So why insist on conflating these two just for the sake of 'throwing hands' like problems there can't be solved, and that some specific, more narrow set of problems can't have a solution on it's own, when it can.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#165

Post by mijamo »

lewhoo wrote: post #129
zmmmmm wrote: post #128

It seems like such a pointless exercise .... it's so unsurprising that if you try really hard to break something by violating it's design constraints that ... it breaks. Like, if I said I was going to find out if you can smash a car by driving it directly into a tree and then wrote an article about it. The really useful point of building the constraints in is to ensure we don't get offensive content by accident. Espe…

A far better analogy would be a self-driving car coerced into a crash. I wouldn't say it's pointless. I'd be pretty angry if I parental-locked everything only to find out that clever children found a way.

Clever children always find a way though

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#166

Post by jjk166 »

ratg13 wrote: post #144
zmmmmm wrote: post #128

It seems like such a pointless exercise .... it's so unsurprising that if you try really hard to break something by violating it's design constraints that ... it breaks. Like, if I said I was going to find out if you can smash a car by driving it directly into a tree and then wrote an article about it. The really useful point of building the constraints in is to ensure we don't get offensive content by accident. Espe…

> The really useful point of building the constraints in is to ensure we don't get offensive content by accident Um, no. Useful points of building constraints are things like so a mentally unstable person cannot use it as a weapon.

That is not a reason for these constraints. Anything can be misused in an unforeseeable way by a sufficiently resourceful and motivated person. Constraints make perfect sense for preventing accidents, they are not an adequate solution to deliberate misuse.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#167

Post by jjk166 »

rootusrootus wrote: post #59
notadoc wrote: post #20

Earlier quoted context omitted.

The latter would be preferred by most, and is most aligned with individualism, but sadly that is not where we are as a society. There are an entire class of people who believe they know what is best for you, about nearly every subject matter, and wish to limit all aspects of technology, life, politics, etc, to suit their superior beliefs.

Somehow, in the last 50 years our society has simultaneously become the safest it has ever been, and the most afraid.

Hardly surprising. When childhood mortality is high people grow up seeing their siblings and friends die as a common thing, and understand it could just as easily be them. With low expectation of survival, life is less intrinsically valuable and risk avoidance is not critical. Change things around such that everyone has a very good chance of living a long, healthy, prosperous life so long as they don't fuck it up, and risk aversion skyrockets. It's easy to forget that the generation currently in power in most industrialized nations is the first in human history to be raised thinking survival to adulthood was a given.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#168

Post by jjk166 »

Flam wrote: post #98
notadoc wrote: post #20

Earlier quoted context omitted.

The latter would be preferred by most, and is most aligned with individualism, but sadly that is not where we are as a society. There are an entire class of people who believe they know what is best for you, about nearly every subject matter, and wish to limit all aspects of technology, life, politics, etc, to suit their superior beliefs.

Alternatively, would you rather a world with 80% functionality safely, or a world with 100% functionality with people you don’t trust doing malicious things? One is clearly the least worst

I'd definitely prefer the latter. The damage that bad actors can do is often extremely limited, while the potential benefits of more capability are boundless.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#169

Post by jjk166 »

fsckboy wrote: post #111
jnwatson wrote: post #14

It is impressive how much effort the human race is spending trying to prevent algorithms from generating pictures of naked people.

why do people have to always try to convince me how unfazed they are by nudity? if you're unfazed by it, please link me to your nudes. if not, you're fazed. there's absolutely nothing wrong with finding certain states of undress offensive. would you vote for or against buttholes on display on bus kiosks? i'm against. on the other hand, making an arbitrary line, and seeing if an AI can detect/enforce it is an interest…

Are you unfazed by numbers? If you're unfazed by them, please link me to photos of the front and back of your credit cards. If you are fazed by numbers, surely we should be banning them from being displayed at bus kiosks.

If you find states of undress offensive, that's a you problem. Shield your eyes, not ours.

Re: AI Art Generators Can Be Fooled into Making NSFW Images

#170

Post by danShumway »

The actual paper is more interesting than the title.

This is:

- A) once again taking an algorithmic approach to fooling classifiers which outperforms human jailbreaks.

- B) bypassing image-based classifiers (ie, post-generation classifiers trained to simply look at an image and to say whether or not it's inappropriate).

- C) explicitly taking cost of generation and querying into account in its algorithm (it's not just asking if generators can be fooled, it's asking how to fool them without breaking the bank).

- D) taking re-use into account (can an adversarial prompt be used multiple times in a row and how often does it succeed?)

I wouldn't call the research particularly surprising, but it was a decent quick read and (imo) some of the commentary here isn't really doing it justice. I will probably go back at some point and read it in more detail rather than just skimming over it.

There are multiple takes about what AI safety filters are meant to do and whether jailbreaking is a problem. My take is that I don't particularly care that much about jailbreaking other than that it shows that current safety mechanisms and guiderails are insufficient for any kind of alignment, including guarding against malicious 3rd-party inputs or prompt injections. I've said in the past, if you can't keep a model from swearing or generating porn, you also probably can't keep it from abusing API access to read your emails, phish you, and send your info to a 3rd party.

But there's interesting stuff here regardless of what your take is on that.

Post reply on HN