Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT – Dalle3 System Prompt

twitter.com

Post reply on HN
Print view View original

121–127 of 127 posts

Re: ChatGPT – Dalle3 System Prompt

#121

Post by fzzzy »

chalsprhebaodu wrote: post #89
sebzim4500 wrote: post #30

Earlier quoted context omitted.

In my experience you kind of just have to lower your standards. i.e. if your system prompt is followed 90% of the time that still a win vs not using one.

I would be happy with 10%. I imagine my problem is using ChatGPT with GPT4 rather than the api. I have had a custom prompt with a mix of various requests listed below, worded many different ways, different combinations, etc. and ChatGPT will happily ignore most of them. - Don’t apologize. - Don’t make changes to the (code, draft, etc) that are not requested. - If I question something about your response to a prompt,…

You may get better results if you phrase all requests positively instead of using "don't". You even have a double negative in your second example.

Re: ChatGPT – Dalle3 System Prompt

#122

Post by a_wild_dandan »

Retric wrote: post #119
famouswaffles wrote: post #117

Earlier quoted context omitted.

a sensory deficit chamber isn't literally depriving people of any stimulus.

There’s no external light, sound, taste, or smell. Most people can always touch themselves or notice the passing of time etc. But it’s possible to be conscious without any external sensation.

You've exclusively described contexts with tons of active stimuli (temperature, interoception, balance, and frankly most of the other senses you named).

But more importantly, suppose we grant that humans function independently of stimuli. Why does that matter? How does this premise imply anything about an agent's capacity for internal experience? In the counterfactual where our brains don't work when surgically placed in life-support vats, does that mean our prior experiences weren't real?

I'm genuinely so confused at this connection between subjective experience and the necessity of stimulus.

Re: ChatGPT – Dalle3 System Prompt

#123

Post by Retric »

a_wild_dandan wrote: post #122
Retric wrote: post #119

Earlier quoted context omitted.

There’s no external light, sound, taste, or smell. Most people can always touch themselves or notice the passing of time etc. But it’s possible to be conscious without any external sensation.

You've exclusively described contexts with tons of active stimuli (temperature, interoception, balance, and frankly most of the other senses you named). But more importantly, suppose we grant that humans function independently of stimuli. Why does that matter? How does this premise imply anything about an agent's capacity for internal experience? In the counterfactual where our brains don't work when surgically place…

Reacting to stimulus doesn’t require subjective experience. A light switch reacts to stimulus.

Reacting without stimulus does. A stopwatch maintains an internal state, the neural networks used by LLM’s don’t.

Someone who starts lucid dreaming can have zero awareness of their body and still do stuff like make up a story which they then recall after waking up.

PS: Balance over all but very brief periods depends on noticing your body weight pressing on something this is one of the reasons people can get disoriented under water. Temperature can be lost track of for similar reasons, rapid changes are noticeable but slowly moving in the neural region ~30-36 °C and all people can detect is a lack of extreme heat or cold not some objective temperature.

Re: ChatGPT – Dalle3 System Prompt

#124

Post by Grimblewald »

NikolaNovak wrote: post #6

So if these are remotely real... And purely as a user of chatgpt not as an ai/ml/nn person... Don't instructions like this weaken the strength of output? Even when request doesn't directly conflict, there are probably myriad valid use cases when instructions will weakly contradict the request. Plus, doesn't it inject inaccuracy into the chain - e.g. it's assuming model confidently knows which artists are 100yo etc. W…

I find that in many cases the most recent things get more attention than other things.

e.g. for the following two approaches

1. intro, instruction, large body of text to work on

2. intro, large body of text to work on, instruction

I find that the second method gets desirable output far more consistently. It could be this would then mean if there are conflicting instructions, the second instruction will simply over-ride the first. This general behavior is also how prompt injection style jailbreaks like DAN work. You're using later contradictory instruction to bring about behavior explicitly forbidden.

Re: ChatGPT – Dalle3 System Prompt

#125

Post by chalsprhebaodu »

fzzzy wrote: post #121
chalsprhebaodu wrote: post #89

Earlier quoted context omitted.

I would be happy with 10%. I imagine my problem is using ChatGPT with GPT4 rather than the api. I have had a custom prompt with a mix of various requests listed below, worded many different ways, different combinations, etc. and ChatGPT will happily ignore most of them. - Don’t apologize. - Don’t make changes to the (code, draft, etc) that are not requested. - If I question something about your response to a prompt,…

You may get better results if you phrase all requests positively instead of using "don't". You even have a double negative in your second example.

Those weren’t the verbatim instructions, and as mentioned, I tried phrasing them many different ways. That said, I don’t recall necessarily trying to phrase every instruction positively, so I’ll try that. Thank you!

Re: ChatGPT – Dalle3 System Prompt

#126

Post by RC_ITR »

famouswaffles wrote: post #118
RC_ITR wrote: post #114

Earlier quoted context omitted.

Again, we are discussing “a common pre-prompt” that you say has probability 1 of showing the system prompt… You are saying there’s some feature of this model that deterministically returns the system prompt and then you pivot to saying that RL could never prevent something from happening. I am saying it’s very easy to use RL to get a model to return a convincing but wrong answer about a system prompt. Then end.

You were wrong. Just admit it and go on with your day. This is what you said. >I am prone to believe that OpenAI, and organization who’s lead is centered on RL more than anything else, is quite good at getting it’s models not to spit out competitively sensitive information I specifically replied it is not possible to prevent a model from spitting this information out. I didn't pivot to anything. >I am saying it’s ver…

This comment is not in the spirit of Hacker News.

I was trying to co-learn by discussing with you and you turned it into something very ugly.

Please do that literally anywhere else on the Internet.

We clearly disagree, but I know have no idea how to move the conversation forward, which is a shame, because maybe you do have something to teach me, though I have no way of knowing at this point.

Re: ChatGPT – Dalle3 System Prompt

#127

Post by l33tman »

Retric wrote: post #87
l33tman wrote: post #77

Earlier quoted context omitted.

Sure, but apart from the detail that you can make them pause by not feeding them words, you can't technically argue that they lack all those things. They are stateful in the sense that they see what they write, so they can keep their inner plan and state in that way across word-iterations. They for sure work differently than a human brain, but without further pretty deep analysis you can't really claim that they can'…

Sometimes, type enough tokens and they no longer have any prior words written by the LLM in their context. Similarly the algorithms would still happily respond if some different and potentially completely unrelated LLM wrote the prior responses. LLM’s are really best thought of as improv actors. The prompt is in effect just the current skit being preformed. The intentions of the character being played doesn’t imply t…

I agree, there is a current technical limit in the token context but that's a limit in practice, not in theory.

There are plenty of wrapper tools around LLMs that cleverly use the token window to keep a longer "state of mind", overall strategies currently executing etc. With varying degrees of success, I should say... but still, it's kind of analogous to a human executing a strategy with intentions.

Post reply on HN