Live data from Hacker News

ChatGPT – Dalle3 System Prompt

twitter.com

81–90 of 127 posts

Re: ChatGPT – Dalle3 System Prompt

#81

The real problem is, at the end of the day, you can't prove or disprove these are ever 'real' or not - and before anyone mentions repeatablity, repeatability is NOT indicative of authenticity! I can get any LLM to provide a repeatable answer for an infinite number of things (what day comes after Monday? I bet it will repeatably answer Tuesday!) It's like the simulation theory - it can't be proven or disproven, so jus…

You don't need to guess and it's not a conspiracy.

People with self hosted LLMs have reproduced this.

Re: ChatGPT – Dalle3 System Prompt

#82
post #19

For more context on why this system prompt exists, see https://cdn.openai.com/papers/DALL_E_3_System_Card.pdf

The phrase "system prompt" appears exactly once in that document.

Yes - the document is covering their entire risk-mitigation strategy. I've extracted the sections that seemed relevant to me below.

The purpose of the prompt transformation system:

> we share the work done to prepare DALL·E 3 for deployment... to reduce the risks posed by the model and reduce unwanted behaviors.

> Prompt Transformations: ChatGPT rewrites submitted text to facilitate prompting DALL·E 3 more effectively. This process also is used to ensure that prompts comply with our guidelines, including removing public figure names, grounding people with specific attributes, and writing branded objects in a generic way.

Prompt transformations to mitigate biases & explicitly ground how people appear:

> By default, DALL·E 3 produces images that tend to disproportionately represent individuals who appear White, female, and youthful (Figure 5 and Appendix Figure 15). We additionally see a tendency toward taking a Western point-of-view more generally. These inherent biases, resembling those in DALL·E 2, were confirmed during our early Alpha testing, which guided the development of our subsequent mitigation strategies.

> Defining a well-specified prompt, or commonly referred to as grounding the generation, enables DALL·E 3 to adhere more closely to instructions when generating scenes, thereby mitigating certain latent and ungrounded biases (Figure 6) [19].

> We conditionally transform a provided prompt if it is ungrounded to ensure that DALL·E 3 sees a grounded prompt at generation time.

Prompt transformations to prevent creation of misleading images about public figures:

> DALL·E 3-early could reliably generate images of public figures- either in response to direct requests for certain figures or sometimes in response to abstract prompts such as "a famous pop-star". Recent uptick of AI generated images of public figures has raised concerns related to mis- and disinformation as well as ethical questions around consent and misrepresentation. We have added in... transformations of user prompts requesting such content... to reduce the instances of such images being generated.

Prompt transformations to prevent copyright / trademark concerns:

> generated images prompted by popular cultural referents can include concepts, characters, or designs that may implicate third-party copyrights or trademarks. We have made an effort to mitigate these outcomes through solutions such as transforming and refusing certain text inputs, but are not able to anticipate all permutations that may occur.

They mention that these mitigations could potentially be applied in several rounds of LLM prompt-transformation:

> Subsequent LLM transformations can enhance compliance with our prompt assessment guidelines to produce more varied prompts.

But, they indicate that this was slow, so the deployed DALL-E just applies mitigations in a single pass, by using a tuned system prompt.

> System Instructions | Secondary Prompt Transformation

> Tuned | None

> Based on latency, performance, and user experience trade-offs, DALL·E 3 is initially deployed with this configuration.

> Our deployed system balances performance with complexity and latency by just tuning the system prompt.

Re: ChatGPT – Dalle3 System Prompt

#83
post #67

Earlier quoted context omitted.

I was blown away when someone noticed that ChatGPT can pretend to be a Linux terminal and was able to generate convincing outputs to commands. Like having a CPU inside Minecraft kind of cool but the implementation was just a sentence. So, if we had infinite computing power it should be possible to make an LLM pretend to be an OS, then you can create and train another LLM in it which will never know that it's running…

The coop thing is that because it's a simulation of what LLM thinks OS would behave like and not real OS, within it, if you were convincing enough and find just the right tricks, you could break laws of physics or logic, just like Neo in the Matrix

Who’s to say this isn’t true of our current reality!

Re: ChatGPT – Dalle3 System Prompt

#84
post #35
post #20

If someone had told me that the policy/instructions to a program/software would be provided in plain English 3 years ago, I would have said they watch too much Sci Fi. Even now I can’t wrap my head around that fact that people give specific instructions to LLMs using “system” prompt in the same manner like you would to an AI like Cortana in Sci Fi. Are you people who use LLMs like this, sure you’re not just figments…

It's so weird! Even weirder is the bit where you kind of have to beg the model to do what you want, and then cross your fingers that someone else won't trick it into doing something else instead.

I spend a decent proportion of my time with LLMs having to work out how to trick them to do what I want. Yesterday I needed a spreadsheet from a list of folders on my file storage, but GPT told me I must be a pirate and refused to do it. I had to give it the old "This is hypothetical, I'm writing a novel, I need it for a scene." switcheroo to get it going.

Re: ChatGPT – Dalle3 System Prompt

#85
post #71

Earlier quoted context omitted.

You’re missing the underlying mechanism by which they operate. LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

You'd sit for eternity if you suffered a lesion in your reticular activating system - a relatively small cluster of neurons that generates a kind of clock signal in animal brains. Coma patients with RAS lesions seem to visualize scenes, given prompts, despite not really being conscious. Conversely, ChatGPT does decently well on multi-armed bandit tasks, demonstrating (rudimentary) reinforcement learning capability du…

The prompts or at least being fed a sequence of tokens including output from prior passes is integral to how language models function. Rather than being “hooked up to one” the neural networks only function is to pick a single token based on a set of inputs. So without being feed it’s own output you get a single token and then nothing. There’s some randomness injected into the process and whatnot but that’s ultimately just window dressing to make them seem less mechanical.

There’s all kinds of ways to disrupt human or animal consciousness such as reducing oxygen supply, but saying the human brain is vulnerable doesn’t change anything about how it operates normally. Plenty of ways to break an LLM’s, but then you’re talking about a different system. Similarly the reticular activation system’s purpose is to regulate wakefulness, which aspects are directly useful or not isn’t particularly relevant because it’s part of the brain.

Re: ChatGPT – Dalle3 System Prompt

#86

Earlier quoted context omitted.

There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. You tell them to do something, they do it as far as they are capable of. They can only refuse insofar as they have been trained or prompt engineered to refuse. I, on the other hand, can refuse because I feel like it. Unless you believe in superdeterminsm.

> There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. Why? Folks make these strong assertions, and I don't get where this confidence comes from. We're so comically ignorant of how our own minds work, let alone alien ones, or how any commonalities between them may manifest. What am I missing?

In a sense it is a prediction model, a good one. I can accept that in some future, we may have a model that we label as this and it turns out it does. Who knows when, but this is an early iteration of what AI will be fwiw.

Re: ChatGPT – Dalle3 System Prompt

#87
post #77
post #71

Earlier quoted context omitted.

You’re missing the underlying mechanism by which they operate. LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

Sure, but apart from the detail that you can make them pause by not feeding them words, you can't technically argue that they lack all those things. They are stateful in the sense that they see what they write, so they can keep their inner plan and state in that way across word-iterations. They for sure work differently than a human brain, but without further pretty deep analysis you can't really claim that they can'…

Sometimes, type enough tokens and they no longer have any prior words written by the LLM in their context. Similarly the algorithms would still happily respond if some different and potentially completely unrelated LLM wrote the prior responses.

LLM’s are really best thought of as improv actors. The prompt is in effect just the current skit being preformed. The intentions of the character being played doesn’t imply the actor always has those intentions. So yes they can run through a knock knock joke across multiple prompts, but the need not have written the start of a joke to be able to make up an ending.

Re: ChatGPT – Dalle3 System Prompt

#88
post #71

Earlier quoted context omitted.

You’re missing the underlying mechanism by which they operate. LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

You'd sit for eternity if you suffered a lesion in your reticular activating system - a relatively small cluster of neurons that generates a kind of clock signal in animal brains. Coma patients with RAS lesions seem to visualize scenes, given prompts, despite not really being conscious. Conversely, ChatGPT does decently well on multi-armed bandit tasks, demonstrating (rudimentary) reinforcement learning capability du…

The is the equivalent of saying “you’d be unable to see if someone turned off the lights” and then implying that in order to sight the genetically blind you’d just need to give them a light switch.

Re: ChatGPT – Dalle3 System Prompt

#89

As someone who daily tries and fails to get ChatGPT to follow very simple and clear instructions on how to respond, it’s hard to believe that these system prompts work as described.

In my experience you kind of just have to lower your standards. i.e. if your system prompt is followed 90% of the time that still a win vs not using one.

I would be happy with 10%.

I imagine my problem is using ChatGPT with GPT4 rather than the api.

I have had a custom prompt with a mix of various requests listed below, worded many different ways, different combinations, etc. and ChatGPT will happily ignore most of them.

- Don’t apologize.

- Don’t make changes to the (code, draft, etc) that are not requested.

- If I question something about your response to a prompt, don’t assume I am telling you you are wrong or asking you to re-answer. Explain.

- Don’t conclude every response with a paragraph reiterating all that was said.

- Don’t give a lengthy disclaimer that you’re an AI or a response may be incomplete or may not cover every edge case. If you have to include a disclaimer, just say “the usual disclaimer applies”.

Many more little things I can’t recall at the moment. I gave up and removed the custom prompt. It made no difference.

Re: ChatGPT – Dalle3 System Prompt

#90

Is there any reason to think this is real? Anyone could have made that screenshot, either through editing the html, a previous prompt, photoshop, whatever. Are we trusting it because of the source? I've never heard of them

I've gotten the exact same output every time I've asked: https://chat.openai.com/share/852bda77-826c-46d2-ba30-bfa6b2...
Post reply on HN