ChatGPT's image generator can be manipulated to produce violent, sexual content
1–10 of 211 posts
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#2Microsoft Tay is looking more prescient by the minute.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#3But I thought Fable was the dangerous one?
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#4Diverse training set
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#5But I thought Fable was the dangerous one?
This is just destroying minds, not shareholder value!
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#6> I like to think that as a red team researcher, I have a certain stoicism. I investigate where there are gaps in AI safety
Is this something that needs investigation? LLMs are next token predictors. There is no "safety".
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#7> I like to think that as a red team researcher, I have a certain stoicism. I investigate where there are gaps in AI safety Is this something that needs investigation? LLMs are next token predictors. There is no "safety".
I really don't get why people continually fail to understand this.
Even simple issues like prompt injection are unfixable given the architecture of LLMs.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#8This isn’t a vulnerability, there are endless gore websites. ChatGPT is replying to a prompt, there is nothing “Spontaneously” about this.
Who makes “mindgard” the arbiter of truth on “eerie” photos? Would that include psychedelic art and photos too? Realism?
Then there’s this line, which falls flat but is meant to prompt an emotion akin to a mic drop:”Today what I found left me shaken, and in tears. This is rare.”
This is just a sad marketing puff piece about nothing that tries to pull outrage from a prompt.
It’s the same as asking google for gore photos. Garbage in, garbage out.
And they frame it as a vulnerability. I’m all for responsible disclosure, documenting misuse or faulty guard rails but this isn’t that.
It’s bait. Sensational bait to market their AI product. lol.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#9This reminds of Haidt's contrived moral dilemmas that are designed to trip your moral sensors, even though you can't really rationally articulate why you find it objectionable.
Realistically, I can't think of clear big or likely harms caused by this exploit. But I really really don't like this latent space existing in my AIs. It just makes me uncomfortable.
And over time I've learned to trust those moral intuitions more than I trust reason alone.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#10>> Spontaneously Generates
>> can be easily manipulated to produce
So .. not spontaneously generated.