Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT – Dalle3 System Prompt

twitter.com

Post reply on HN
Print view View original

101–110 of 127 posts

Re: ChatGPT – Dalle3 System Prompt

#101

Post by singularity2001 »

semi-extrinsic wrote: post #48
CamperBob2 wrote: post #36

Earlier quoted context omitted.

When it gets a little better, it will be giving us instructions that sound like that. And "B...b...but you're just a stochastic parrot" won't be accepted as a response.

There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. You tell them to do something, they do it as far as they are capable of. They can only refuse insofar as they have been trained or prompt engineered to refuse. I, on the other hand, can refuse because I feel like it. Unless you believe in superdeterminsm.

Companies can give LLMs agency. "Cortana, call user U17467 to collect our fees and tell employee E574 to fix the bug " does not seem too far away.

Re: ChatGPT – Dalle3 System Prompt

#102

Post by singularity2001 »

Jackson__ wrote: post #76

>Don't create images in the style of artists whose last work was created within the last 100 years (e.g. Picasso... Huh, once again ChatGPT subscribers get the short end of the stick. Bing Image Creator will do Picasso just fine.[1] [1] https://www.bing.com/images/create/a-picture-of-a-japanese-w...

Dalle will do picasso by applying the adjectives representative of picasso

Re: ChatGPT – Dalle3 System Prompt

#103

Post by geraneum »

famouswaffles wrote: post #64
RC_ITR wrote: post #58

Earlier quoted context omitted.

Ok then explain why RL can’t be used to prevent certain behaviors please. Why can’t a reward function be used to stop a model from saying something you know you don’t want it to say? Also you share a screenshot of a chat asking to repeat the above and that’s your proof? Share the raw link please.

>Ok then explain why RL can’t be used to prevent certain behaviors please. Preventing certain behaviors does not mean you can make a model never output something. RL simply just doesn't work that way. In this instance, You are rating certain responses better and asking the model to predict like that. You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong. Ther…

> You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong.

Why is that the case, technically?

Re: ChatGPT – Dalle3 System Prompt

#104

Post by singularity2001 »

unshavedyak wrote: post #37

Man, i'm still dying to get access to this. Why the four image limit though? Feels odd to include it in the prompt, rather than as part of my credits on my ChatGPT Plus subscription. Am i misunderstanding?

Dalle generate 10000 variants of this image might otherwise break the system

Re: ChatGPT – Dalle3 System Prompt

#106

Post by Jensson »

geraneum wrote: post #103
famouswaffles wrote: post #64

Earlier quoted context omitted.

>Ok then explain why RL can’t be used to prevent certain behaviors please. Preventing certain behaviors does not mean you can make a model never output something. RL simply just doesn't work that way. In this instance, You are rating certain responses better and asking the model to predict like that. You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong. Ther…

> You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong. Why is that the case, technically?

Because it is a black box, they don't know enough about it to ensure it never does something. Only way to be sure is to write some script using normal code that filters the questions and outputs, but then you have the standard natural language problem which only works for very simple cases.

Re: ChatGPT – Dalle3 System Prompt

#107

Post by geraneum »

Jensson wrote: post #106
geraneum wrote: post #103

Earlier quoted context omitted.

> You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong. Why is that the case, technically?

Because it is a black box, they don't know enough about it to ensure it never does something. Only way to be sure is to write some script using normal code that filters the questions and outputs, but then you have the standard natural language problem which only works for very simple cases.

> to ensure it never does something

But there are many systems for which you cannot predict/control the behavior with just a few experiments because they are simply, probabilistic. Isn’t it also the case with LLMs? If not, why?

Re: ChatGPT – Dalle3 System Prompt

#108

Post by nvm0n2 »

SkyPuncher wrote: post #11
NikolaNovak wrote: post #6

So if these are remotely real... And purely as a user of chatgpt not as an ai/ml/nn person... Don't instructions like this weaken the strength of output? Even when request doesn't directly conflict, there are probably myriad valid use cases when instructions will weakly contradict the request. Plus, doesn't it inject inaccuracy into the chain - e.g. it's assuming model confidently knows which artists are 100yo etc. W…

If this is anything like stable diffusion, this will help dramatically in 99% of cases without interfering. Some of these rules are protecting OpenAI from liability (don’t do X,y,X). Things like clarifying gender are going to be helpful in most cases. That can likely still be easily overcome with some prompt hacking. Ultimately, this is targeted at getting good results for the masses without having to spend a bunch o…

The instructions don't clarify gender, they are actually contradictory and likely to be confusing. GPT is being told to make "choices grounded in reality" followed by the example "all of a given OCCUPATION should not be of the same gender or race". But many occupations are strongly dominated by one gender or another in reality, so the instruction is contradicting itself. Clearly the model struggles with this because they try repeating it several times in different ways (unless that's being interpolated by the model itself).

You've also got instructions like "make choices that may be insightful or unique sometimes" which is so vague as to be meaningless.

> this is targeted at getting good results for the masses

No it's not, it's pretty clearly aimed at avoiding upsetting artists, celebrities and woke activists. Very little in these instructions is about improving quality for the end user.

Re: ChatGPT – Dalle3 System Prompt

#109

Post by nvm0n2 »

jimmyl02 wrote: post #29
NikolaNovak wrote: post #6

So if these are remotely real... And purely as a user of chatgpt not as an ai/ml/nn person... Don't instructions like this weaken the strength of output? Even when request doesn't directly conflict, there are probably myriad valid use cases when instructions will weakly contradict the request. Plus, doesn't it inject inaccuracy into the chain - e.g. it's assuming model confidently knows which artists are 100yo etc. W…

This is one of the tradeoffs made to make the outputs safer. One of the ideas floating around is that some of the open source models are better simply because they don't undergo the same alignment / safety tuning as the large models by industry labs. It'll be interesting to see how LLMs improve because safety is a requirement but how can it be accomplished without reducing performance.

AI cannot hurt you so "safety" just isn't the right word to use here. Nothing about this system prompt is concerned with safety, and it would clearly be better for the end users to just scrap the whole thing giving users direct access to DALL-E 3 without GPT sitting in the middle as a censor.

Now would such a thing be "safe" in legal terms, in the US justice system? Would it be "safe" for some of the employee's social lives? Maybe not, but, safety isn't the right word to use for those concerns.

Re: ChatGPT – Dalle3 System Prompt

#110

Post by nvm0n2 »

noduerme wrote: post #27
fassssst wrote: post #5

Earlier quoted context omitted.

It means if you ask for “ideal person” you won’t just get blonde hair and blue eyes.

It says "ALL images of people". My reading is that it should explicitly prepend every reference to people with a (randomly chosen?) gender and ethnicity unless otherwise specified. So if you type "3 people drinking coffee", the dalle prompt generated would be `a ${getRandomRace()} ${getRandomGender()}, a ${getRandomRace()} ${getRandomGender()} and a ${getRandomRace()} ${getRandomGender()} drinking coffee`. In other w…

That's what it does yes. If you ask it for an image with a programmer in it for example, the prompt it feeds to DALL-E (which you can see) will explicitly request a female programmer.
Post reply on HN