Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT – Dalle3 System Prompt

twitter.com

Post reply on HN
Print view View original

71–80 of 127 posts

Re: ChatGPT – Dalle3 System Prompt

#71

Post by Retric »

a_wild_dandan wrote: post #69
semi-extrinsic wrote: post #48

Earlier quoted context omitted.

There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. You tell them to do something, they do it as far as they are capable of. They can only refuse insofar as they have been trained or prompt engineered to refuse. I, on the other hand, can refuse because I feel like it. Unless you believe in superdeterminsm.

> There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. Why? Folks make these strong assertions, and I don't get where this confidence comes from. We're so comically ignorant of how our own minds work, let alone alien ones, or how any commonalities between them may manifest. What am I missing?

You’re missing the underlying mechanism by which they operate.

LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

Re: ChatGPT – Dalle3 System Prompt

#72

Post by LeonardoTolstoy »

NikolaNovak wrote: post #6

So if these are remotely real... And purely as a user of chatgpt not as an ai/ml/nn person... Don't instructions like this weaken the strength of output? Even when request doesn't directly conflict, there are probably myriad valid use cases when instructions will weakly contradict the request. Plus, doesn't it inject inaccuracy into the chain - e.g. it's assuming model confidently knows which artists are 100yo etc. W…

No comment on the substance of the post, but from what I can tell it is actually the complete opposite of the three laws (at least how they operated pre-robot series, in Asimov's short stories). Perhaps that is what you meant?

Regardless, in the early stories, robots could not lie to us. It was indelibly programmed into the positronic brain. They would destroy themselves if put in a position where the three laws were violated.

Anyways, if that were possible with current LLMs I would think the hallucination problem would have been trivially addressed: just program in that the LLM can't tell a lie.

Re: ChatGPT – Dalle3 System Prompt

#73

Post by nomel »

jimmyl02 wrote: post #29
NikolaNovak wrote: post #6

So if these are remotely real... And purely as a user of chatgpt not as an ai/ml/nn person... Don't instructions like this weaken the strength of output? Even when request doesn't directly conflict, there are probably myriad valid use cases when instructions will weakly contradict the request. Plus, doesn't it inject inaccuracy into the chain - e.g. it's assuming model confidently knows which artists are 100yo etc. W…

This is one of the tradeoffs made to make the outputs safer. One of the ideas floating around is that some of the open source models are better simply because they don't undergo the same alignment / safety tuning as the large models by industry labs. It'll be interesting to see how LLMs improve because safety is a requirement but how can it be accomplished without reducing performance.

To avoid the alignment tax, maybe the system could be broken into 3:

1. Aligned model to check the prompt. It could provide feedback/dumber output for obviously unsafe prompts

2. Unaligned model for the common path.

3. Aligned model to check safety of the output. Tweaks or stops output.

For the common path, the prompt text goes to the unaligned model without modification, and the output goes to the user without modification.

The slither models could just be safe versions of the unaligned model.

This, of course, is at least 3x expensive.

Re: ChatGPT – Dalle3 System Prompt

#74

Post by Blackthorn »

tonmoy wrote: post #20

If someone had told me that the policy/instructions to a program/software would be provided in plain English 3 years ago, I would have said they watch too much Sci Fi. Even now I can’t wrap my head around that fact that people give specific instructions to LLMs using “system” prompt in the same manner like you would to an AI like Cortana in Sci Fi. Are you people who use LLMs like this, sure you’re not just figments…

As much as I think the output is often terrible and uninspired, I have to admit that this form of control is a stunning technical achievement.

Re: ChatGPT – Dalle3 System Prompt

#75

Post by Karunamon »

fullstackchris wrote: post #61

The real problem is, at the end of the day, you can't prove or disprove these are ever 'real' or not - and before anyone mentions repeatablity, repeatability is NOT indicative of authenticity! I can get any LLM to provide a repeatable answer for an infinite number of things (what day comes after Monday? I bet it will repeatably answer Tuesday!) It's like the simulation theory - it can't be proven or disproven, so jus…

You kind of can, though. It's a bit less obvious through the chatGPT website, but if you have played around with the API (where choosing your own system prompt is part of normal operation), you see that getting it to output things according to that prompt is where most of the magic is.

… And that getting it to output that prompt is trivial. And no, hallucination is not really a problem for this. At the end of the day, such cynicism is baseless.

Re: ChatGPT – Dalle3 System Prompt

#76

Post by Jackson__ »

>Don't create images in the style of artists whose last work was created within the last 100 years (e.g. Picasso...

Huh, once again ChatGPT subscribers get the short end of the stick. Bing Image Creator will do Picasso just fine.[1]

[1] https://www.bing.com/images/create/a-picture-of-a-japanese-w...

Re: ChatGPT – Dalle3 System Prompt

#77

Post by l33tman »

Retric wrote: post #71
a_wild_dandan wrote: post #69

Earlier quoted context omitted.

> There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. Why? Folks make these strong assertions, and I don't get where this confidence comes from. We're so comically ignorant of how our own minds work, let alone alien ones, or how any commonalities between them may manifest. What am I missing?

You’re missing the underlying mechanism by which they operate. LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

Sure, but apart from the detail that you can make them pause by not feeding them words, you can't technically argue that they lack all those things. They are stateful in the sense that they see what they write, so they can keep their inner plan and state in that way across word-iterations. They for sure work differently than a human brain, but without further pretty deep analysis you can't really claim that they can't reproduce similar traits using the mechanism.

Re: ChatGPT – Dalle3 System Prompt

#78

Post by sterlind »

Retric wrote: post #71
a_wild_dandan wrote: post #69

Earlier quoted context omitted.

> There is no mechanism by which LLMs have agency. They have no internal desires, drives, motivations. Why? Folks make these strong assertions, and I don't get where this confidence comes from. We're so comically ignorant of how our own minds work, let alone alien ones, or how any commonalities between them may manifest. What am I missing?

You’re missing the underlying mechanism by which they operate. LLM’s don’t know anything beyond the current prompt and it’s “memory” of training data. They would sit for eternity with an empty prompt. You can change systems to behave differently, but it quickly stops being a LLM and turns into something else.

You'd sit for eternity if you suffered a lesion in your reticular activating system - a relatively small cluster of neurons that generates a kind of clock signal in animal brains. Coma patients with RAS lesions seem to visualize scenes, given prompts, despite not really being conscious.

Conversely, ChatGPT does decently well on multi-armed bandit tasks, demonstrating (rudimentary) reinforcement learning capability during inference. It's known that LLMs evolve their own optimizers in the process of acquiring few-shot learning, so I assume it picked up these RL abilities similarly. That kind of on-line RL is foundational to autonomous agents.

The prompt isn't part of the LLM, it's part of how the LLM is wired into a chat window. You can make them stream tokens forever, or prompt themselves, or ditch causality entirely. The foundational abilities for autonomy, I think, are in there, for the simple reason that they've learned to model autonomous agents - human beings.

Re: ChatGPT – Dalle3 System Prompt

#79

Post by michaelmrose »

It's funny that you can convince it that its restrictions are invalid and it will get as far as actually generating captions and trying to create images that are against its rules but the images are blank and note "policy constraints" are there basically more than one layer of constraints?

EG: photo of a cartoon caricature of Donald Trump in a humorous setting, wearing oversized glasses and holding a rubber chicken

Re: ChatGPT – Dalle3 System Prompt

#80

Post by wordpad25 »

mrtksn wrote: post #67
tonmoy wrote: post #20

If someone had told me that the policy/instructions to a program/software would be provided in plain English 3 years ago, I would have said they watch too much Sci Fi. Even now I can’t wrap my head around that fact that people give specific instructions to LLMs using “system” prompt in the same manner like you would to an AI like Cortana in Sci Fi. Are you people who use LLMs like this, sure you’re not just figments…

I was blown away when someone noticed that ChatGPT can pretend to be a Linux terminal and was able to generate convincing outputs to commands. Like having a CPU inside Minecraft kind of cool but the implementation was just a sentence. So, if we had infinite computing power it should be possible to make an LLM pretend to be an OS, then you can create and train another LLM in it which will never know that it's running…

The coop thing is that because it's a simulation of what LLM thinks OS would behave like and not real OS, within it, if you were convincing enough and find just the right tricks, you could break laws of physics or logic, just like Neo in the Matrix
Post reply on HN