Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT – Dalle3 System Prompt

twitter.com

Post reply on HN
Print view View original

111–120 of 127 posts

Re: ChatGPT – Dalle3 System Prompt

#111

Post by Retric »

sterlind wrote: post #99
Retric wrote: post #85

Earlier quoted context omitted.

The prompts or at least being fed a sequence of tokens including output from prior passes is integral to how language models function. Rather than being “hooked up to one” the neural networks only function is to pick a single token based on a set of inputs. So without being feed it’s own output you get a single token and then nothing. There’s some randomness injected into the process and whatnot but that’s ultimately…

No, it doesn't pick a single token based on a set of inputs. It predicts a probability distribution for the next token given the previous tokens. That's why techniques like beam search and Viterbi work so well - you don't have to commit to the next token at each step. And temperature (what I assume you mean by "randomness injected") isn't "window dressing," it fundamentally gives better results because LMs model prob…

There’s plenty of different ways you can use the output of those functions to feed a new token sequences back to the model, but you can only feed a specific token not the full probability distribution from a prior run.

As to randomness that’s simply one approach, there’s deterministic approaches that have their own advantages. What randomness provides over them is avoiding always responding to the same opening in the same way as that’s quite off-putting.

Re: ChatGPT – Dalle3 System Prompt

#112

Post by Retric »

a_wild_dandan wrote: post #91
Retric wrote: post #71

Earlier quoted context omitted.

You’re missing the underlying mechanism by which they operate. LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

I don't see a connection between how an agent works and what it experiences. Sure, depriving myself or LLMs of all neural activity results in uninteresting behavior. How does this fact buy us insight into how agents feel in other circumstances?

A human in a sensory deficit chamber still thinks demonstrating they aren’t necessarily purely reacting the the current stimulus.

An LLM is only reacting to its current stimulus.

Re: ChatGPT – Dalle3 System Prompt

#113

Post by ignoramous »

tonmoy wrote: post #20

If someone had told me that the policy/instructions to a program/software would be provided in plain English 3 years ago, I would have said they watch too much Sci Fi. Even now I can’t wrap my head around that fact that people give specific instructions to LLMs using “system” prompt in the same manner like you would to an AI like Cortana in Sci Fi. Are you people who use LLMs like this, sure you’re not just figments…

> Even now I can't wrap my head around that fact that people give specific instructions to LLMs using "system" prompt in the same manner like you would...

"Natural Language Processing" now that it works, to the extent that it does, doesn't seem short of magic.

Re: ChatGPT – Dalle3 System Prompt

#114

Post by RC_ITR »

famouswaffles wrote: post #64
RC_ITR wrote: post #58

Earlier quoted context omitted.

Ok then explain why RL can’t be used to prevent certain behaviors please. Why can’t a reward function be used to stop a model from saying something you know you don’t want it to say? Also you share a screenshot of a chat asking to repeat the above and that’s your proof? Share the raw link please.

>Ok then explain why RL can’t be used to prevent certain behaviors please. Preventing certain behaviors does not mean you can make a model never output something. RL simply just doesn't work that way. In this instance, You are rating certain responses better and asking the model to predict like that. You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong. Ther…

Again, we are discussing “a common pre-prompt” that you say has probability 1 of showing the system prompt…

You are saying there’s some feature of this model that deterministically returns the system prompt and then you pivot to saying that RL could never prevent something from happening.

I am saying it’s very easy to use RL to get a model to return a convincing but wrong answer about a system prompt.

Then end.

Re: ChatGPT – Dalle3 System Prompt

#115

Post by semi-extrinsic »

a_wild_dandan wrote: post #69
semi-extrinsic wrote: post #48

Earlier quoted context omitted.

There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. You tell them to do something, they do it as far as they are capable of. They can only refuse insofar as they have been trained or prompt engineered to refuse. I, on the other hand, can refuse because I feel like it. Unless you believe in superdeterminsm.

> There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. Why? Folks make these strong assertions, and I don't get where this confidence comes from. We're so comically ignorant of how our own minds work, let alone alien ones, or how any commonalities between them may manifest. What am I missing?

Precisely this, we have almost no idea about how our minds work, how they obtain agency and free will and all that jazz.

Large language models in our current paradigm developing agency would be like 16th century alchemists inventing nuclear fusion reactors.

Re: ChatGPT – Dalle3 System Prompt

#116

Post by Racing0461 »

nvm0n2 wrote: post #110
noduerme wrote: post #27

Earlier quoted context omitted.

It says "ALL images of people". My reading is that it should explicitly prepend every reference to people with a (randomly chosen?) gender and ethnicity unless otherwise specified. So if you type "3 people drinking coffee", the dalle prompt generated would be `a ${getRandomRace()} ${getRandomGender()}, a ${getRandomRace()} ${getRandomGender()} and a ${getRandomRace()} ${getRandomGender()} drinking coffee`. In other w…

That's what it does yes. If you ask it for an image with a programmer in it for example, the prompt it feeds to DALL-E (which you can see) will explicitly request a female programmer.

Can you provide the prompt?

Re: ChatGPT – Dalle3 System Prompt

#117

Post by famouswaffles »

Retric wrote: post #112
a_wild_dandan wrote: post #91

Earlier quoted context omitted.

I don't see a connection between how an agent works and what it experiences. Sure, depriving myself or LLMs of all neural activity results in uninteresting behavior. How does this fact buy us insight into how agents feel in other circumstances?

A human in a sensory deficit chamber still thinks demonstrating they aren’t necessarily purely reacting the the current stimulus. An LLM is only reacting to its current stimulus.

a sensory deficit chamber isn't literally depriving people of any stimulus.

Re: ChatGPT – Dalle3 System Prompt

#118

Post by famouswaffles »

RC_ITR wrote: post #114
famouswaffles wrote: post #64

Earlier quoted context omitted.

>Ok then explain why RL can’t be used to prevent certain behaviors please. Preventing certain behaviors does not mean you can make a model never output something. RL simply just doesn't work that way. In this instance, You are rating certain responses better and asking the model to predict like that. You can make it more likely to refuse a request but the idea that you can guarantee it won't is completely wrong. Ther…

Again, we are discussing “a common pre-prompt” that you say has probability 1 of showing the system prompt… You are saying there’s some feature of this model that deterministically returns the system prompt and then you pivot to saying that RL could never prevent something from happening. I am saying it’s very easy to use RL to get a model to return a convincing but wrong answer about a system prompt. Then end.

You were wrong. Just admit it and go on with your day.

This is what you said.

>I am prone to believe that OpenAI, and organization who’s lead is centered on RL more than anything else, is quite good at getting it’s models not to spit out competitively sensitive information

I specifically replied it is not possible to prevent a model from spitting this information out. I didn't pivot to anything.

>I am saying it’s very easy to use RL to get a model to return a convincing but wrong answer about a system prompt.

No it's not.

Re: ChatGPT – Dalle3 System Prompt

#119

Post by Retric »

famouswaffles wrote: post #117
Retric wrote: post #112

Earlier quoted context omitted.

A human in a sensory deficit chamber still thinks demonstrating they aren’t necessarily purely reacting the the current stimulus. An LLM is only reacting to its current stimulus.

a sensory deficit chamber isn't literally depriving people of any stimulus.

There’s no external light, sound, taste, or smell. Most people can always touch themselves or notice the passing of time etc. But it’s possible to be conscious without any external sensation.

Re: ChatGPT – Dalle3 System Prompt

#120

Post by TeMPOraL »

thethimble wrote: post #83
wordpad25 wrote: post #80

Earlier quoted context omitted.

The coop thing is that because it's a simulation of what LLM thinks OS would behave like and not real OS, within it, if you were convincing enough and find just the right tricks, you could break laws of physics or logic, just like Neo in the Matrix

Who’s to say this isn’t true of our current reality!

The secret trick to make a prayer work is to prompt-inject God.
Post reply on HN