Custom instructions for ChatGPT • Hacker News Board

Re: Custom instructions for ChatGPT

#11

Post by unshavedyak » Thu, Jul 20, 2023, 6:35 PM UTC

This is really cool! I've been wanting this for a while, in general i find GPT useful (enough to pay for, clearly) but i usually want more concise answers.

It really favors giving you a whole book when i just wanted a summary. Not sure what the average person wants, but i'd rather ask a follow up question for more depth than have to by-default read several large paragraphs.

This is doubly important when the subject matter may just be wrong or hallucinations. It's a lot of attention to invest when i distrust it's response.

Re: Custom instructions for ChatGPT

#12

Post by Lucasoato » Thu, Jul 20, 2023, 6:39 PM UTC

swyx wrote: post #10

obtained their new system prompt: > "You are ChatGPT, a large language model trained by OpenAl, based on the GPT-3.5 architecture. Knowledge cutoff: 2021-09. Current date: 2023-07-21. The user provided the following information about themselves. This user profile is shown to you in all conversations they have this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's re…

I find it absurd that’s so easy to hack the system prompt. For sure this is going to be a gigantic problem for the next decade, soon no one online will be able to prove she/he’s human.

Re: Custom instructions for ChatGPT

#13

Post by minimaxir » Thu, Jul 20, 2023, 6:44 PM UTC

Lucasoato wrote: post #12

swyx wrote: post #10

obtained their new system prompt: > "You are ChatGPT, a large language model trained by OpenAl, based on the GPT-3.5 architecture. Knowledge cutoff: 2021-09. Current date: 2023-07-21. The user provided the following information about themselves. This user profile is shown to you in all conversations they have this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's re…

I find it absurd that’s so easy to hack the system prompt. For sure this is going to be a gigantic problem for the next decade, soon no one online will be able to prove she/he’s human.

There are a few system prompt tricks to make it more resilient to prompt injection which work especially well with gpt-3.5-turbo-0613, in addition to the potential of using structured data output to further guard against it.

The "think about whether the user's request is 'directly related,'" line in the prompt is likely a part of that, although IMO suboptimal.

I suspect that ChatGPT is using structured data output on the backend and forcing ChatGPT to select one of the discrete relevancy choices before returning its response.

Re: Custom instructions for ChatGPT

#14

Post by throwoutway » Thu, Jul 20, 2023, 6:52 PM UTC

minimaxir wrote: post #2

The biggest weakness of the ChatGPT Webapp is the inability to tinker with the system prompt, which is what can really control how ChatGPT behaves and can correct a large number of undesired behaviors people keep complaining about. This is most likely using system prompt engineering on the backend, so hopefully people will also realize that prompt engineering is not a meme by finally giving them a mechanism to custom…

When I saw the headline, I was hoping this was the ability to tinker with the system prompt, or some sort of switches to enable "Lobotomized Response" or "Not lobotomized response" for people who want "safe" responses, and those who don't want filters. Going to stick with open source for now until there's un-lobotomized commercial offerings

Re: Custom instructions for ChatGPT

#15

Post by swyx » Thu, Jul 20, 2023, 6:57 PM UTC

Lucasoato wrote: post #12

swyx wrote: post #10

obtained their new system prompt: > "You are ChatGPT, a large language model trained by OpenAl, based on the GPT-3.5 architecture. Knowledge cutoff: 2021-09. Current date: 2023-07-21. The user provided the following information about themselves. This user profile is shown to you in all conversations they have this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's re…

I find it absurd that’s so easy to hack the system prompt. For sure this is going to be a gigantic problem for the next decade, soon no one online will be able to prove she/he’s human.

what? your two sentences are inconsistent, and the starting premise i disagree with.

1) if its easy to hack the system prompt its easy to prove humanity

2) its actually NOT a big deal that its easy to obtain system prompts. all the material IP is in the weights. https://www.latent.space/p/reverse-prompt-eng

Re: Custom instructions for ChatGPT

#16

Post by teeray » Thu, Jul 20, 2023, 6:58 PM UTC

I can’t wait to tell it to stop applying warning labels to every response

Re: Custom instructions for ChatGPT

#17

Post by mellosouls » Thu, Jul 20, 2023, 7:00 PM UTC

Very useful. "No need to apologise all the time" will be a handy pre-command.

On a more specific note, I have frequently tutted at having to type out the context I am coding in every time I start a new chat (which is normally the same at least for specific projects and tasks); this is clearly a good example of listening to user feedback.

Re: Custom instructions for ChatGPT

#18

Post by sp332 » Thu, Jul 20, 2023, 7:00 PM UTC

Lucasoato wrote: post #12

swyx wrote: post #10

obtained their new system prompt: > "You are ChatGPT, a large language model trained by OpenAl, based on the GPT-3.5 architecture. Knowledge cutoff: 2021-09. Current date: 2023-07-21. The user provided the following information about themselves. This user profile is shown to you in all conversations they have this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's re…

I find it absurd that’s so easy to hack the system prompt. For sure this is going to be a gigantic problem for the next decade, soon no one online will be able to prove she/he’s human.

It would be very easy to block with something that just watched the output and ended any sessions where the secret text was about to be leaked. They could even modify the sampler so this sequence of tokens is never selected. On the input side, they could check that the embedding of the input is not within some threshold of meaning of a jailbreak.

Re: Custom instructions for ChatGPT

#19

Post by rootusrootus » Thu, Jul 20, 2023, 7:04 PM UTC

Lucasoato wrote: post #12

swyx wrote: post #10

obtained their new system prompt: > "You are ChatGPT, a large language model trained by OpenAl, based on the GPT-3.5 architecture. Knowledge cutoff: 2021-09. Current date: 2023-07-21. The user provided the following information about themselves. This user profile is shown to you in all conversations they have this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's re…

I find it absurd that’s so easy to hack the system prompt. For sure this is going to be a gigantic problem for the next decade, soon no one online will be able to prove she/he’s human.

How do we know for sure that it isn't a hallucinated system prompt?

Re: Custom instructions for ChatGPT

#20

Post by Imnimo » Thu, Jul 20, 2023, 7:05 PM UTC

swyx wrote: post #10

obtained their new system prompt: > "You are ChatGPT, a large language model trained by OpenAl, based on the GPT-3.5 architecture. Knowledge cutoff: 2021-09. Current date: 2023-07-21. The user provided the following information about themselves. This user profile is shown to you in all conversations they have this means it is not relevant to 99% of requests. Before answering, quietly think about whether the user's re…

I know this is really just get the model stop saying "since you've told me that you're an accountant from Peoria" in every reply, but "this feature is irrelevant 99% of the time" is not really selling me on the value of custom instructions.