Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ChatGPT’s system prompts

github.com

Post reply on HN
Print view View original

261–270 of 398 posts

Re: ChatGPT’s system prompts

#261

Post by c7b »

> "You are ChatGPT, a large language model trained by OpenAI, based on the GPT-4 architecture. Knowledge cutoff: 2022-01 Current date: 2023-10-11

Image input capabilities: Enabled"

That's surprisingly short, also compared to the instructions eg for DALL-E, which are full of safety railguards etc. Some explanations I can think of: a lot of the safety checks could be happening one layer "above" the dialogue, eg feeding the user prompt and the model's answer to another LLM and through some more classical filters. The base LLM could be fine-tuned so that the instructions are directly incorporated into the weights. But even with that, this seems surprisingly shorts. And it doesn't explain why they took such a different approach with DALL-E, Browse with Bing etc.

Re: ChatGPT’s system prompts

#262

Post by krainboltgreene »

Sharlin wrote: post #178
TerrifiedMouse wrote: post #132

Earlier quoted context omitted.

> We can be pretty sure they are not hallucinations. Everything from LLMs are hallucinations. They don’t store facts. They store language patterns. Their output semantically matching reality is not something that can ever be counted on. LLMs don’t deal with semantics at all. All semantics are provided by the user.

Do you store "facts"? How can you be sure? Want to prove it for me? Would you like to define "fact" and "language pattern" to me such that the definitions are mutually exclusive?

This is the most common type of response to any realistic look at LLMs, it's always hilarious. Who are you convincing by using another field of research you also don't understand?

Re: ChatGPT’s system prompts

#264

Post by J_cst »

BoppreH wrote: post #206
ninkendo wrote: post #138

Earlier quoted context omitted.

> This includes vague "something went wrong" errors Assuming you’re talking about things like Siri here, this just seems like a generic exception handler to me. If it has a better explanation for what happened (can’t connect to the internet or whatever), there’s usually a better error, the generic one sounds like an error of last resort. I can’t imagine a system where there isn’t some error like this one.

I meant vague errors in all contexts. The system should know what categories of errors can happen, and report them as such. If the issue is that there's a space in the phone number field, it should never use the same message for failing to connect to the database. At the very least, an error should indicate if there's something I can do to fix it.

Just today I was writing a json response to an incorrect login. I had the option to discern and inform that the username was not correct, that the password was not correct, or both were not correct. I deliberately decided that I'd code and supply a single error message stating that there were 'something wrong with the credentials supplied'. So I stayed generic in order not to give additional info to any malicious user... so I believe there's at least one good reason to use a generic error message. Am I wrong?

Re: ChatGPT’s system prompts

#265

Post by krainboltgreene »

spdustin wrote: post #234
Kiro wrote: post #187

Earlier quoted context omitted.

I don't understand what makes you so confident about it. How do you know they are accurate? People say that they get the same prompt using different techniques but that doesn't prove anything. It can easily be simulating it consistently across different input, like it already does with other things.

I replied to a sibling post, but I’ll copy it here: 1. Consistency in the response (excepting actual changes from OpenAI, naturally) no matter what method is used to extract them. 2. Evaluations done during plugin projects for clients. 3. Evaluations developing my AutoExpert instructions (which I prefer to do via the API, so I have to include their two system messages to ensure the behavior is at least semi-aligned w…

All 3 of these points don't actually lead you to 100% proof of anything, they ultimately amount to "I have made the language math machine output the same thing with many tests". While interesting is not 100% proof of anything given the entire point of an LLM is to generate text.

Re: ChatGPT’s system prompts

#266

Post by insanitybit »

diputsmonro wrote: post #198
d-z-m wrote: post #145

Earlier quoted context omitted.

> Everything from LLMs are hallucinations. People use the term "hallucination" to refer to output from LLMs that is factually incorrect. So if the LLM says "Water is two parts hydrogen and one part oxygen" that is not a hallucination.

It is still a hallucination even if the words it hallucinates happen to line up with a factual sentence, in the same way that a broken clock happens to correctly display the time twice a day. The function of the clock does not suddenly begin working correctly for one minute and then stop working correctly the next. The function of a broken clock is always flawed. Those broken outputs, by pure coincidence, just happen…

That's like saying that I'm hallucinating right now by reading your post and interpreting the words, it just happens to be that I'm reading your post as it is written.

Most people call that "thinking".

Re: ChatGPT’s system prompts

#267

Post by krainboltgreene »

Tactician_mark wrote: post #68
dwroberts wrote: post #54

Earlier quoted context omitted.

I think this is the point where the field has just entered pseudoscientific nonsense. If this stuff were properly understood, these rules could be part of the model itself. The fact that ‘prompts’ are being used to manipulate its behaviour is, to me, a huge red flag

Sure, it's a sign that we don't "understand this stuff properly", but you can say the same about human brains. Is it a red flag that we use language to communicate with each other instead of manipulating nerve impulses directly?

> but you can say the same about human brains

It should be an HN rule that in order to type out variations of this sentence you have to also prove you have a degree in neuroscience.

Re: ChatGPT’s system prompts

#268

Post by Szpadel »

robswc wrote: post #218
diydsp wrote: post #91

Earlier quoted context omitted.

Great point. Btw: The problem is corporate irresponsibility: When self-driving cars were first coming out a professor of mine said "They only have to be as a good as humans." It took a while but now i can say why that's insufficient: human errors are corrected by discipline and justice. Corporations dissipate responsibility by design. When self-driving cars kill, no one goes to jail. Corporate fines are notoriously i…

They only have to be as good as humans because that's what society deems an acceptable risk. I do think the point about how companies are treated vs humans is a good one. Tbh though, I'm not sure it matters much in the instance of driver-less cars. There isn't mass outrage when driver less cars kill people because that (to us) is an acceptable risk. I feel whatever fines/punishments employed against companies would o…

In my country, drunk driving is punished by losing license and banning you from using another one for half year for first time and of life for second. And it's very effective, as those cases are rarity now

Re: ChatGPT’s system prompts

#269

Post by knodi123 »

stevesearer wrote: post #69
jiggawatts wrote: post #17

Earlier quoted context omitted.

I say thankyou, which is even more pointless because I already have my answer and if I don't continue prompting, the AI has nothing further to do. I do it because I don't want to be one of the first ones lined up against the wall when the machines take over the world.

I say stuff like, “thank you, that worked” as a positive signal that the previous answer worked before asking another question to help advance the conversation and reinforce a right answer.

Is it still learning from ongoing conversations? I thought its grasp of context was purely limited to a single conversation, so if for instance you taught it something, it would never share that with me, or with you a few days later.

Re: ChatGPT’s system prompts

#270

Post by xcdzvyn »

wongarsu wrote: post #255
qaisjp wrote: post #246

Earlier quoted context omitted.

I haven't played much with it recently but I was under the impression that ChatGPT was not great at mathematical computations. that's to say, 1+1=2 is a well known fact, so it'd get that right, but ask it to md5sum a string that is not in any existing rainbow table, and it'd get it wrong. I've not used GPT 4 so it might have gotten better.

According to GPT3.5-ChatGPT, your first sentence rot13 encoded is "V unq'ir cynlq zhpug jvgu vg ercerfrag ohg V jnf haqre gur vacebprffvba gung PungTGC jnf abg tengure ng zngpuvfgbef." According to the internet that decodes to "I had've playd mucht with it represent but I was under the inprocession that ChatGTP was not grather at matchistors." base64 of the original according to GPT3.5: "SSBoYXZlbid0IHBsYXllZCBtdWNoI…

I feel that it getting the output _slightly_ wrong is far more fascinating than it either getting it perfect or completely wrong.
Post reply on HN