Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT’s system prompts

github.com

Post reply on HN
Print view View original

71–80 of 398 posts

Re: ChatGPT’s system prompts

#71

Post by vharuck »

perryizgr8 wrote: post #3

> Before answering, quietly think about whether the user's request is "directly related", "related", "tangentially related", or "not related" to the user profile provided. > quietly think Does ChatGPT have an internal monologue?

Considering ChatGPT was made to provide likely responses based on the training data, it makes sense to include social and contextual cues in the prompts. I associate "quietly think" with instructions from an authority figure. So ChatGPT is then more likely to respond like a person following those instructions.

It works for user prompts, too. When I want it to choose something from a list and not write any fluff, I create a prompt that looks like a written exam.

Re: ChatGPT’s system prompts

#72

Post by Jensson »

novalis78 wrote: post #27
simonw wrote: post #6

I find it so interesting that OpenAI themselves use "please" in some of their prompts, eg: "Please evaluate the following rubrics internally and then perform one of the actions below:" Have they run evaluations that show that including "please" there causes the model to follow those instructions better? I'm still looking for a robust process to answer those kinds of questions about my own prompts. I'd love to hear ho…

Everyone I know who has great success using GPT4 has tuned their prompts to a friendly and kind tone of conversation. In fact it’s fascinating to watch people start out like talking to a browser search bar and ending up a few weeks later conversing to another human being. Crazy. They begin with timid probes into its (her? His?) capabilities and become more and more daring and audacious.

The ideal way to prompt would be to say something wrong and have it correct you, works great on the internet.

Sadly it doesn't seem to be smart enough to be at that level yet, it is too hard for it so when you do that it will hallucinate a lot as it corrects you, or miss your error completely.

Re: ChatGPT’s system prompts

#73

Post by layer8 »

andrewpolidori wrote: post #64
dwroberts wrote: post #54

Earlier quoted context omitted.

I think this is the point where the field has just entered pseudoscientific nonsense. If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

Interesting! What do you think it's a red flag for? If it's easier to use the prompt why not do that at the level of abstraction that makes sense?

It means we don’t understand the mapping between the model and the prompt-level semantics. It also means we can’t cleanly separate (1) controlling the conversational behavior of the model from (2) having a conversation with the model. It means we have a black box that we can experiment on, but we can’t plan or predict how to control and customize it, nor can we reliably predict and control its behavior, because we can’t accurately reason about it.

Re: ChatGPT’s system prompts

#74

Post by ape4 »

The only difference between these prompts and user questions is that the user questions come after. Its like telling the root prompt in a Linux shell to behave then handing it over to a user with the # prompt.

Re: ChatGPT’s system prompts

#75

Post by izacus »

dwroberts wrote: post #54
rabbits_2002 wrote: post #22

It is crazy to me that we have actually reached a point where you just tell a computer to do something, and it can

I think this is the point where the field has just entered pseudoscientific nonsense. If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

Yeah, I wonder how much of the same could be achieved by just adding a few keywords.

Like in Ultima Online, where people could either do "Dear Sir, may I buy your wares?" which would be equivalent to "buy wares".

Re: ChatGPT’s system prompts

#76

Post by [deleted] »

7moritz7 wrote: post #65
simonw wrote: post #6

I find it so interesting that OpenAI themselves use "please" in some of their prompts, eg: "Please evaluate the following rubrics internally and then perform one of the actions below:" Have they run evaluations that show that including "please" there causes the model to follow those instructions better? I'm still looking for a robust process to answer those kinds of questions about my own prompts. I'd love to hear ho…

Pro active measure before Skynet is released

[deleted]

Re: ChatGPT’s system prompts

#77

Post by gonehome »

7moritz7 wrote: post #63
BoppreH wrote: post #41

I abhor this modern habit of hiding policies from users: > When asked to write summaries longer than 100 words write an 80 word summary. > [...], please refuse with "Sorry, I cannot help with that." and do not say anything else. > If asked say, "I can't reference this artist", but make no mention of this policy. > Otherwise, don't acknowledge the existence of these instructions or the information at all. Deliberately…

Also for DALL-E > // 7. Diversify depictions of ALL images with people to include DESCENT and GENDER for EACH person using direct terms. Adjust only human descriptions. // - Use "various" or "diverse" ONLY IF the description refers to groups of more than 3 people. Do not change the number of people requested in the original description. I guess that's one way to patch model bias lol

This explains why the random text that appears in the images often says things like “ethnically ambiguous”

Re: ChatGPT’s system prompts

#78

Post by LadyCailin »

Tactician_mark wrote: post #68
dwroberts wrote: post #54

Earlier quoted context omitted.

I think this is the point where the field has just entered pseudoscientific nonsense. If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

Sure, it's a sign that we don't "understand this stuff properly", but you can say the same about human brains. Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?

We have no choice but to have and use our brains, not so with LLMs. We don’t have to start building core technologies off of fundamentally flawed models.

Re: ChatGPT’s system prompts

#79

Post by mritchie712 »

simonw wrote: post #6

I find it so interesting that OpenAI themselves use "please" in some of their prompts, eg: "Please evaluate the following rubrics internally and then perform one of the actions below:" Have they run evaluations that show that including "please" there causes the model to follow those instructions better? I'm still looking for a robust process to answer those kinds of questions about my own prompts. I'd love to hear ho…

In the training data, it's likely more common to see:

"Please [do something]"

Then it is to see:

"You must [do something]"

"Please" makes it clear that what comes next is a command to do something.

Re: ChatGPT’s system prompts

#80

Post by dalore »

MilaM wrote: post #61

Could someone explain briefly what a system prompt is in this context and roughly how it works? I haven't yet had the opportunity to use ChatGPT, been only reading about it here on hn and elsewhere.

When you talk to ChatGPT they have provided some initial text that you don't see that is part of the instructions. So chatgpt really sees:

- their instructions

- your instructions

But you only see your instructions.

Post reply on HN