I’m surprised how long these prompts are, I wonder at what point is the diminishing returns.
Given the token budget they consume, the returns are literally diminishing. ;)
Anthropic publishes the 'system prompts' that make Claude tick
121–130 of 290 posts
Re: Anthropic publishes the 'system prompts' that make Claude tick
#122Earlier quoted context omitted.
> No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. I would say even a foundation model, without supervised instruction tuning, and without RLHF, understands text quite well. It just predicts the most likely continuation of the prompt, but to do so effectively, it arguably has to understand what the text means.
If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…
But it messes something so simple up because it doesn't actually understand things.
Meanwhile on the human side: https://neuroscienceresearch.wustl.edu/how-your-mind-plays-t...
Re: Anthropic publishes the 'system prompts' that make Claude tick
#123Claude has been pretty great. I stood up an 'auto-script-writer' recently, that iteratively sends a python script + prompt + test results to either GPT4 or Claude, takes the output as a script, runs tests on that, and sends those results back for another loop. (Usually took about 10-20 loops to get it right) After "writing" about 5-6 python scripts this way, it became pretty clear that Claude is far, far better - if only because I often ended up using Claude to clean up GPT4's attempts. GPT4 would eventually go off the rails - changing the goal of the script, getting stuck in a local minima with bad outputs, pruning useful functions - Claude stayed on track and reliably produced good output. Makes sense that it's more expensive.
Edit: yes, I was definitely making sure to use gpt-4o
Re: Anthropic publishes the 'system prompts' that make Claude tick
#124Earlier quoted context omitted.
Odd how many of those instructions are almost always ignored (eg. "don't apologize," "don't explain code without being asked"). What is even the point of these system prompts if they're so weak?
It's common for neural networks to struggle with negative prompting. Typically it works better to phrase expectations positively, e.g. “be brief” might work better than ”do not write long replies”.
But surely Anthropic knows better than almost anyone on the planet what does and doesn't work well to shape Claude's responses. I'm curious why they're choosing to write these prompts at all.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#125Earlier quoted context omitted.
Given that it's a big next-word-predictor, I think it has to do with matching the training data. For the vast majority of text out there, someone's personality, goals, etc. are communicated via a narrator describing how thing are. (Plays, stories, almost any kind of retelling or description.) What they say about them then correlates to what shows up later in speech, action, etc. In contrast, it's extremely rare for s…
But #2 is a good example of "show, don't tell" which is arguably a better writing style. Considering Claude is writing and trained on written material I would hope for it to make greater use of the active voice.
> But #2 is a good example of "show, don't tell" which is arguably a better writing style.
I think both examples are almost purely "tell", where the person who went to the doctor is telling the listener discrete facts about their doctor. The difference is that the second retelling is awkward, unrealistic, likely a lie, and just generally not how humans describe certain things in English.
In contrast, "showing" the doctor's traits might involve retelling a longer conversation between patient and doctor which indirectly demonstrates how the doctor responds to words or events in a careful way, or--if it were a movie--the camera panning over the doctor's Certificate Of Carefulness on the office wall, etc.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#126This part seems to imply that facial recognition is on by default: Claude always responds as if it is completely face blind. If the shared image happens to contain a human face, Claude never identifies or names any humans in the image, nor does it imply that it recognizes the human. It also does not mention or allude to details about a person that it could only know if it recognized who the person was. Instead, Claud…
I doubt facial recognition is a switch turned "on", rather its vision capabilities are advanced enough that it can recognize famous faces. Why would they build in a separate facial recognition algorithm? Seems to go against the whole ethos of a single large multi-modal model that many of these companies are trying to build.
Not necessarily famous, but faces existing in training data or false positives making generalizations about faces based on similar characteristics to faces in training data. This becomes problematic for a number of reasons, e.g., this face looks dangerous or stupid or beautiful, etc.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#127Earlier quoted context omitted.
And "kinda" is an understatement. It understands you very well, perhaps even better than the average human would. (Average humans often don't understand jargon.)
> It understands you very well No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. To truly conclude it understands things, one needs to show some internal cause and effect, to disprove a Chinese Room scenario. https://en.wikipedia.org/wiki/Chinese_room
When it listens to your prompt and responds accordingly, that's an instance of undertanding. The magic of LLMs is on the input side, not the output.
Searle's point wasn't relevant when he made it, and it hasn't exactly gotten more insightful with time.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#128> Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. Meanwhile my every respond from Claude: > Certainly! [...] Same goes with > It avoids starting its responses with “I’m sorry” or “I apologize” and every time I spo…
It's possible it reduces the rate but doesn't fix it.
This did make me wonder how much of their training data is support emails and chat, where they have those phrases as part of standard responses.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#129Earlier quoted context omitted.
> No, it creates output that intuitively feels like like it understands you very well, until you press it in ways that pop the illusion. I would say even a foundation model, without supervised instruction tuning, and without RLHF, understands text quite well. It just predicts the most likely continuation of the prompt, but to do so effectively, it arguably has to understand what the text means.
If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…
If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry.
How about if it recognized its limitations with regard to introspecting its tokenization process, and wrote and ran a Python program to count the r's? Would that change your opinion? Why or why not?
Re: Anthropic publishes the 'system prompts' that make Claude tick
#130Why do the three models have different system prompts? and why is Sonnet's longer than Opus'
Prompts tend not to be transferable across different language models