Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

61–70 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#61

Post by SirMaster »

cubefox wrote: post #44
Terr_ wrote: post #27

Earlier quoted context omitted.

> It understands you very well No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. To truly conclude it understands things, one needs to show some internal cause and effect, to disprove a Chinese Room scenario. https://en.wikipedia.org/wiki/Chinese_room

> No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. I would say even a foundation model, without supervised instruction tuning, and without RLHF, understands text quite well. It just predicts the most likely continuation of the prompt, but to do so effectively, it arguably has to understand what the text means.

If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry.

But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this.

If it was truly understanding, then it should be able to understand and figure out how to work around these such limitations in the math.

At least in my opinion.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#62

Post by Bluestein »

Terr_ wrote: post #29
zevv wrote: post #28

Earlier quoted context omitted.

It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of natural language.

LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole unti…

... or - worse even - something you think is what you want, because you know not better, but happens to be a wholy (or - worse - even just subtly partially incorrect) confabulated answer.-

Re: Anthropic publishes the 'system prompts' that make Claude tick

#63

Post by SirMaster »

ano-ther wrote: post #46

This makes me so happy as I find the pseudo-conversational tone of other GPTs quite off-putting. > Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. https://docs.anthropic.com/en/release-notes/system-prompts

If only it actually worked...

Re: Anthropic publishes the 'system prompts' that make Claude tick

#65

Post by CSMastermind »

chilling wrote: post #22

> Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. Meanwhile my every respond from Claude: > Certainly! [...] Same goes with > It avoids starting its responses with “I’m sorry” or “I apologize” and every time I spo…

Same, even when it should not apologize Claude always says that to me.

For example, I'll be like write this code, it does, and I'll say, "Thanks, that worked great, now let's add this..."

It will still start it's reply with "I apologize for the confusion". It's a particularly odd tick of that system.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#66

Post by brookst »

Terr_ wrote: post #29
zevv wrote: post #28

Earlier quoted context omitted.

It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of natural language.

LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole unti…

As a product manager this is largely my experience with developers.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#67

Post by brookst »

JumpCrisscross wrote: post #59
aden1ne wrote: post #42

Earlier quoted context omitted.

This is one of the reasons why I'm paying for Claude and not for ChatGPT. ChatGPT really goes into uncanny valley for me.

> ChatGPT really goes into uncanny valley for me Especially with the exclamation marks, it reads to me the way a stereotypical Silicon Valley bullshitter speaks.

Certainly! I can see why you think that!

Re: Anthropic publishes the 'system prompts' that make Claude tick

#68

Post by brookst »

Terr_ wrote: post #27
cubefox wrote: post #15

Earlier quoted context omitted.

And "kinda" is an understatement. It understands you very well, perhaps even better than the average human would. (Average humans often don't understand jargon.)

> It understands you very well No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. To truly conclude it understands things, one needs to show some internal cause and effect, to disprove a Chinese Room scenario. https://en.wikipedia.org/wiki/Chinese_room

This seems as fruitful as debating whether my car brought me to work today because some connotations of “bring” include volition.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#69

Post by brookst »

SirMaster wrote: post #61
cubefox wrote: post #44

Earlier quoted context omitted.

> No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. I would say even a foundation model, without supervised instruction tuning, and without RLHF, understands text quite well. It just predicts the most likely continuation of the prompt, but to do so effectively, it arguably has to understand what the text means.

If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…

The limitations on processing letters aren’t in the math, they are in the encoding. Language is the map, and concepts are the territory. You may as well complain that someone doesn’t really understand their neighborhood if they can’t find it on a map.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#70

Post by jabroni_salad »

ano-ther wrote: post #46

This makes me so happy as I find the pseudo-conversational tone of other GPTs quite off-putting. > Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. https://docs.anthropic.com/en/release-notes/system-prompts

Unfortunately I suspect that line is giving it a "dont think about pink elephants" problem. Whether or not it acts like that was up to random chance but describing it at all is a positive reinforcement.

It's very evident in my usage anyways. If I start the convo with something like "You are terse and direct in your responses" the interaction is 110% more bearable.

Post reply on HN