Live data from Hacker News

ChatGPT’s system prompts

github.com

141–150 of 398 posts

Re: ChatGPT’s system prompts

#141

Earlier quoted context omitted.

I have seen some people go even further and start up different chats, where in each tab they start by describing the character they want to chat with, and then moving on to talking with it.

Isn't that standard? I only use the API (it's usually cheaper), so I don't know. Chatbox for example lets you configure different personas to talk to.

I have no idea what the norm is. On the website it’s a free text box you can type anything into.

Re: ChatGPT’s system prompts

#142
post #15

Surprised by some of the choices. e.g. for web browsing they're calling it "id" instead of "url". Would have thought that would be clearer for the LLM. Similarly > Keep the conversation flowing. seems like a very human concept. I wonder if they A/B tested these - maybe it does make a difference

The id refers to the id of the quote they extracted (the one with the start and finish “lines”). They are given back to the user’s client as a metadata sidecar to the actual completion.

Re: ChatGPT’s system prompts

#143
post #6

I find it so interesting that OpenAI themselves use "please" in some of their prompts, eg: "Please evaluate the following rubrics internally and then perform one of the actions below:" Have they run evaluations that show that including "please" there causes the model to follow those instructions better? I'm still looking for a robust process to answer those kinds of questions about my own prompts. I'd love to hear ho…

It may be just an impression but ChatGPT used to give me very dry answers bordering on being dismissive and it got better and even enthusiastic when I started using Please... And this has been for technical questions, documentation, etc. I suppose that the Please token filters out some less friendly neuron pathways in the model.

Re: ChatGPT’s system prompts

#144
post #6

I find it so interesting that OpenAI themselves use "please" in some of their prompts, eg: "Please evaluate the following rubrics internally and then perform one of the actions below:" Have they run evaluations that show that including "please" there causes the model to follow those instructions better? I'm still looking for a robust process to answer those kinds of questions about my own prompts. I'd love to hear ho…

A lot of the training data is written in a polite manner so it makes sense to use similar style when asking for a continuation

Re: ChatGPT’s system prompts

#145

Earlier quoted context omitted.

Some of them, like the standard ChatGPT prompt, have been repeatedly retrieved by many people over long time periods, using very different methods. We can be pretty sure they are not hallucinations. And correctly retrieving these prompts lends credence to the claim that you were successful at extracting the other prompts, even though it's not conclusive proof. Of course OpenAI might have a completely different prompt…

> We can be pretty sure they are not hallucinations. Everything from LLMs are hallucinations. They don’t store facts. They store language patterns. Their output semantically matching reality is not something that can ever be counted on. LLMs don’t deal with semantics at all. All semantics are provided by the user.

> Everything from LLMs are hallucinations.

People use the term "hallucination" to refer to output from LLMs that is factually incorrect. So if the LLM says "Water is two parts hydrogen and one part oxygen" that is not a hallucination.

Re: ChatGPT’s system prompts

#147
post #98

Earlier quoted context omitted.

It feels like a Turing Test pass when social engineering is a valid attack.

The real pass will be when ChatGPT calls your bullshit.

Well, if calling bullshit is the real Turing test, then I'm afraid some real people won't be able to pass it: (trigger warning!) https://en.wikipedia.org/wiki/Strip_search_phone_call_scam

Re: ChatGPT’s system prompts

#148

> Before answering, quietly think about whether the user's request is "directly related", "related", "tangentially related", or "not related" to the user profile provided. > quietly think Does ChatGPT have an internal monologue?

It’s more like taking advantage of an emergent property of the model’s attention mechanisms, so that it doesn’t confirm or acknowledge the instruction in its text (or “out loud”) completion.

Re: ChatGPT’s system prompts

#149
post #98

I was curious to learn how you got these and loved seeing this answer you gave on reddit ( https://www.reddit.com/r/OpenAI/comments/176mxj8/comment/k4r... ): >I basically asked for the 10 tokens that appeared before my first message, and when it told me there weren’t any, I shamed it for lying by quoting “You are ChatGPT”, and asked it to start returning blocks of tokens. Each time, I said “Okay, I think I might lear…

It feels like a Turing Test pass when social engineering is a valid attack.

I'm not sure "social engineering" is the right terminology here. Maybe "threatening" or "shaming"?

Re: ChatGPT’s system prompts

#150

It is crazy to me that we have actually reached a point where you just tell a computer to do something, and it can

I think this is the point where the field has just entered pseudoscientific nonsense. If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

I think what you're asking for--this total scientific control and understanding of how the model will behave was never going to happen for any models close to our own intelligence.
Post reply on HN