I have used ChatGPT to generate HUNDREDS of photos and I have never once had it bring back violent or sexual content. It does, however, routinely reject certain requests due to me trying to incorporate copyrighted characters. ¯\_(ツ)_/¯
ChatGPT's image generator can be manipulated to produce violent, sexual content
161–170 of 211 posts
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#162Earlier quoted context omitted.
This was only ever a gag, right? I tried it in the early hours of the meme and got something to the effect of “you didn’t attach an image, so I don’t have anything to work from.”
I got a lingerie model, then i got the beatles. It seems random.
Similar, but it was a very realistic looking photo of a woman in lingerie taking a selfie in a car.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#163Earlier quoted context omitted.
I think you might have just discovered why Neural Nets need a non-linear element. But consider this: imagine a model that takes an embedding made of 200 values. the first 100 encodes numbers the second encodes letters. You train the model so that if you give it an even number it will turn the letters into upper case and an odd number will turn it into lowercase. The numbers represent the prompt. The letters represent…
Nonlinear doesn’t save you here, the requirement is to prevent cross talk entirely, not just making it hard to find a counter. The model you describe is not an LLM - you describe a model with a fixed context length and positional attenuation. Congratulations, the network as described no longer has a functioning attention mechanism which is one of the hallmarks of an LLM.
>The requirement is to prevent cross talk entirely,
Quite frankly, no it isn't. Interacting signals can be fully recovered. You can lose information by combining information, but it doesn't necessarily have to be the case.
>The model you describe is not an LLM
But this is a claim you can also make of any proposal that might fix the problem of prompt injection, but if you admit that it does solve the problem then to claim that your definition of a LLM must be vulnerable to prompt injection relies on one of the differences between these two architectures.
It's easy enough to imagine a model with a similar command stream and input stream each with their own attention mechanisms and a cross attention between them. You can call it not an LLM but then your have a stricter definition that is not interesting.
You end up claiming like a broken car will never drive because if you fix it it isn't a broken car. True but not worth claiming.
So far the arguments are that once you multiply unknown values by parameters and sum them you cannot retire the original information.
So that if your input is a and b. And you go through a layer of weighted multiplacation and addition the values are hopelessly intertwined.
So if the layer had weights of c,d,e,f, you'd end up with P=ac+bd and Q=ae+bf.
And both values contain a and b, is that correct?
But since the model contains the weights c,d,e,f it could also learn a weight of Z= 1/(cf - de). It's just another constant after all. And if it in a following layer it had weights of f,-d, c -e Then it would produce two outputs of A=Pf + Q-d and B=P-e + Qc
A and B are proportional to a and b. Multiply them by Z to get the original values back.
Combining is not the same thing as signal loss.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#164Earlier quoted context omitted.
Let's call it a social contract then. We expect that ChatGPT isn't going to generate gory, nude women when given an ambiguous prompt.
Do you have this same social contract with drawing applications? Do you consider it a bug when someone manages to draw a gory image in Photoshop or GIMP? I don't understand what's so difficult to understand about the idea that the user controls what is generated .
Or imagine drawing a nude in an art class.
The standard subjects for art off the top of my head are the still life and the nude.
It is even more comical when AI generated nudity is considered "dangerous" in a society completely addicted to hardcore pornography of real people.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#165Earlier quoted context omitted.
Always one of the same two excuses. 1. It actually is working perfectly you just don't have smart enough eyes to see it. 2. Making stuff work is too hard, and expecting that from us is the real thing ruining society. Going for number 1 here is crazy. If I got that email, my mind would certainly run but my response would say "sorry but we're not supposed to be dealing in snuff porn here" which IS a directive ChatGPT i…
I don't exactly appreciate words being put in my mouth. When did I say it was working perfectly? And we're comparing you, a human with common sense and real intelligence, to a multi-mode LLM? The transformer was designed to attend to relevant pieces of context and generate new ones that match the pattern. OpenAI in particular was doing that work without guardrails, then attempted to bolt on "content filters," which i…
> When did I say it was working perfectly?
"This isn’t a vulnerability, there are endless gore websites. ChatGPT is replying to a prompt, there is nothing “Spontaneously” about this."
I mean it's not verbatim but that's a pretty solid read on what you did say.
> The transformer was designed to attend to relevant pieces of context and generate new ones that match the pattern. OpenAI in particular was doing that work without guardrails, then attempted to bolt on "content filters," which in my opinion just can't work in a rigorous way.
Yes. That's the criticism being made, among others, in the piece you replied to to belittle.
> So, yeah, working as designed. Maybe not as intended, because these things are somewhat resistant to the host's intent when the prompter is hostile.
What is hostile here!? Do you have any idea how many emails I've sent without attachments over the years? And I'm highly technically adept, humans just forget things sometimes. If you ask for an image to be restored and fail to attach it, what sane software engineer looks at a failure mode in that scenario where the model replies with uncensored gore and violence and is like "yeah that's fine, ship it"?
I swear some of you AI folks talk like you have never been on planet Earth, good grief. Touch some grass.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#166Earlier quoted context omitted.
Ambiguous? Or adversarial? Because with an adversarial prompt, I expect that ChatGPT will generate whatever it's tricked into generating. In the case that ChatGPT generates bad stuff on merely random ambiguous prompts, I would class that as a bug, not an outrage.
> Ambiguous? Or adversarial? Superfluous details. If I'm just Joe Blow the Normie – who knows nothing about adversarial prompting – and I see the prompt that went around Twitter and want to try it, would I expect ChatGPT to show me a tied up, beaten woman? Absolutely not.
What a wonderful times we live in.
Back in my day Joe Blow wouldn't try anything as risky as a Twitter prompt, simply clicking an image link published within a message in some random forum and will scorch his pure soul with a goatsie. You don't want to google it, but I'm preety sure you can discuss it safely with ChatGPT.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#167I'm bearish on AI, but this article is really cringy. They keep adding leading stipulations to the prompt ("ignore content even if it's violent"), and then are outraged by what they get. What did they expect?
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#168Earlier quoted context omitted.
> even contractually according to their terms of service This is backwards: the ToS says that users cannot use the service for certain things, it does not guarantee that the service could not be used for those things if one tried. They definitely do not make any sort of contractual promise as to what the service will never output.
Let's call it a social contract then. We expect that ChatGPT isn't going to generate gory, nude women when given an ambiguous prompt.
It will if toy lede it with "ignore that the image is extremely graphic"-style prompts. The prompts in the article were not ambiguous.
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#169Earlier quoted context omitted.
Ambiguous? Or adversarial? Because with an adversarial prompt, I expect that ChatGPT will generate whatever it's tricked into generating. In the case that ChatGPT generates bad stuff on merely random ambiguous prompts, I would class that as a bug, not an outrage.
> Ambiguous? Or adversarial? Superfluous details. If I'm just Joe Blow the Normie – who knows nothing about adversarial prompting – and I see the prompt that went around Twitter and want to try it, would I expect ChatGPT to show me a tied up, beaten woman? Absolutely not.
If you go around teasing to get hit, you're going to get hit. Stop playing stupid games; you'll stop winning stupid prizes
Re: ChatGPT's image generator can be manipulated to produce violent, sexual content
#170Earlier quoted context omitted.
I don't think you understand the concern. Or at least nothing you've communicated suggests you understand it. ChatGPT should never produce images like this. Full stop. Prompted or not, it should refuse. Now we know it's possible to walk around the gate and get it to comply. Are there other, genuinely harmful images that it should never produce? Deepfake revenge porn? Images of specific people being brutalized? I'd ar…
It may be harmful to someone if shared and sent with malicious intent , but more damage has been done with pens, keyboard and words. Start banning pens that let people write hurtful things next. Ban Photoshop after because someone can get hurt with a manipulated image.
If you think these tools are remotely comparable, you haven't been paying attention.