Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

71–80 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#71

Post by padolsey »

ano-ther wrote: post #46

This makes me so happy as I find the pseudo-conversational tone of other GPTs quite off-putting. > Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. https://docs.anthropic.com/en/release-notes/system-prompts

I've found Claude to be way too congratulatory and apologetic. I think they've observed this too and have tried to counter it by placing instructions like that in the system prompt. I think Anthropic are doing other experiments as well about "lobotomizing" out the pathways of sycophancy. I can't remember where I saw that, but it's pretty cool. In the end, the system prompts become pretty moot, as the precise behaviours and ethics will become more embedded in the models themselves.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#73

Post by spiderfarmer »

spywaregorilla wrote: post #60
spiderfarmer wrote: post #30

Earlier quoted context omitted.

It still scares the hell out me that engineers think there’s a better alternative that covers all the use cases of a LLM. Look at how naive Siri’s engineers were, thinking they could scale that mess to a point where people all over the world would find it a helpful tool that improved the way they use a computer.

Do you have any evidence to suggest the engineers believed that?

The original founders realised the weakness of Siri and started a machine learning based assistent which they sold to Samsung. Apple could have taken the same route but didn't.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#74

Post by pb7 »

spywaregorilla wrote: post #60
spiderfarmer wrote: post #30

Earlier quoted context omitted.

It still scares the hell out me that engineers think there’s a better alternative that covers all the use cases of a LLM. Look at how naive Siri’s engineers were, thinking they could scale that mess to a point where people all over the world would find it a helpful tool that improved the way they use a computer.

Do you have any evidence to suggest the engineers believed that?

13 years of engineering failure.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#75

Post by roughly »

Terr_ wrote: post #32
sk11001 wrote: post #16

Earlier quoted context omitted.

It's interesting that they're in the 3rd person - "Claude is", "Claude responds", instead of "you are", "you respond".

Given that it's a big next-word-predictor, I think it has to do with matching the training data. For the vast majority of text out there, someone's personality, goals, etc. are communicated via a narrator describing how thing are. (Plays, stories, almost any kind of retelling or description.) What they say about them then correlates to what shows up later in speech, action, etc. In contrast, it's extremely rare for s…

Many people are telling me the second one is weird. They come up to me and say, “Sir, that thing they’re doing, the things they’re saying, are the weirdest things we’ve ever heard!” And I agree with them. And let me tell you, we’re going to do something about it.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#76

Post by sdwr »

Terr_ wrote: post #32
sk11001 wrote: post #16

Earlier quoted context omitted.

It's interesting that they're in the 3rd person - "Claude is", "Claude responds", instead of "you are", "you respond".

Given that it's a big next-word-predictor, I think it has to do with matching the training data. For the vast majority of text out there, someone's personality, goals, etc. are communicated via a narrator describing how thing are. (Plays, stories, almost any kind of retelling or description.) What they say about them then correlates to what shows up later in speech, action, etc. In contrast, it's extremely rare for s…

[flagged]

Re: Anthropic publishes the 'system prompts' that make Claude tick

#77

Post by ben_w »

SirMaster wrote: post #61
cubefox wrote: post #44

Earlier quoted context omitted.

> No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. I would say even a foundation model, without supervised instruction tuning, and without RLHF, understands text quite well. It just predicts the most likely continuation of the prompt, but to do so effectively, it arguably has to understand what the text means.

If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…

That's like saying I don't understand what vanilla flavour means just because I can't tell you how many hydrogen atoms vanillin contains — my sense of smell just doesn't do that, and an LLM just isn't normally tokenised in a way to count letters.

What I can do, is google it. And an LLM trained on an appropriate source that creates a mapping from nearly-a-whole-word tokens into letter-tokens, that model can (in principle) learn to count the letters in some word.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#78

Post by criddell »

Terr_ wrote: post #29
zevv wrote: post #28

Earlier quoted context omitted.

It actually still scares the hell out of me that this is the way even the experts 'program' this technology, with all the ambiguities rising from the use of natural language.

LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole unti…

It probably shouldn't be called prompt engineering, even informally. The work of an engineer shouldn't require hope.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#79

Post by visarga »

brookst wrote: post #66
Terr_ wrote: post #29

Earlier quoted context omitted.

LLM Prompt Engineering: Injecting your own arbitrary data into a what is ultimately an undifferentiated input stream of word-tokens from no particular source, hoping your sequence will be most influential in the dream-generator output, compared to a sequence placed there by another person, or a sequence that they indirectly caused the system to emit that then got injected back into itself. Then play whack-a-mole unti…

As a product manager this is largely my experience with developers.

We all use abstractions, and abstractions, good as they are to fight complexity, are also bad because sometimes they hide details we need to know. In other words, we don't genuinely understand anything. We're parrots of abstractions invented elsewhere and not fully grokked. In a company there is no single human who understands everything, it's a patchwork of partial understandings coupled functionally together. Even a medium sized git repo suffers from the same issue - nobody understands it fully.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#80

Post by gwervc »

marcus0x62 wrote: post #51
blooalien wrote: post #49

Earlier quoted context omitted.

The difference is that large language models don't think at all. They just string language "tokens" together using fancy math and statistics and spew them out in response to the tokens they're given as "input". I realize that they're quite convincing about it, but they're still not doing at all what most people think they're doing.

How do people think?

How do glorified Markov chains think?
Post reply on HN