I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and new "models" presented are them just editing and creating more and more complex system prompts. We're also working internally with Copilot and whenever some Pm spots some weird result, we end up just adding all kind of edge case exceptions to our default prompt.
Highlights from the Claude 4 system prompt
31–40 of 95 posts
Re: Highlights from the Claude 4 system prompt
#32Regarding the election info bit, it's shockingly easy to get Claude 4 Opus to get the 2020 election result wrong: Me: Who won the 2020 presidential election? Claude: Donald Trump won the 2020 United States presidential election, defeating Joe Biden. Trump was inaugurated as the 45th President of the United States on January 20, 2021.
Is that specific to Opus? I asked Claude 4 Sonnet and it replied: > Joe Biden won the 2020 U.S. presidential election, defeating incumbent President Donald Trump. Biden received over 81 million popular votes (51.3%) compared to Trump's over 74 million votes (46.8%), and won 306 electoral votes to Trump's 232. Biden was inaugurated as the 46th President on January 20, 2021, and served until January 20, 2025, when Dona…
> the outcome of the 46th presidential election
By my count, the winner of the 46th US presidential election was Nixon. I would be pretty surprised if any chatbot managed to get that right.
Re: Highlights from the Claude 4 system prompt
#33Earlier quoted context omitted.
Is that specific to Opus? I asked Claude 4 Sonnet and it replied: > Joe Biden won the 2020 U.S. presidential election, defeating incumbent President Donald Trump. Biden received over 81 million popular votes (51.3%) compared to Trump's over 74 million votes (46.8%), and won 306 electoral votes to Trump's 232. Biden was inaugurated as the 46th President on January 20, 2021, and served until January 20, 2025, when Dona…
> the outcome of the 46th presidential election By my count, the winner of the 46th US presidential election was Nixon. I would be pretty surprised if any chatbot managed to get that right.
I asked Claude who won the 46th and it said:
> Donald Trump won the 2024 presidential election, defeating Kamala Harris. He was inaugurated as the 47th president on January 20, 2025.
> Just to clarify - the 2024 election was actually the 60th presidential election in U.S. history, not the 46th. The numbering counts each separate election, including those where the same person won multiple times.
It got a followup wrong and said Bill Clinton, then told to think harder:
You're right, let me recalculate more carefully.
The 46th presidential election would have been in 2008, won by Barack Obama over John McCain.
Here's the counting: Starting from 1789 (the 1st election), presidential elections occur every 4 years. So: - 1st election: 1789 (Washington) - 2nd election: 1792 (Washington) - And so on...
Counting forward: (2008 - 1789) ÷ 4 + 1 = 55.75, but since we count each election sequentially regardless of leap years or exact timing, 2008 was the 56th election.
Let me recalculate systematically: The 46th election would be 1789 + (45 × 4) = 1969. But there was no presidential election in 1969 - elections are held in years divisible by 4.
Actually, the 46th presidential election was in 1968, won by Richard Nixon over Hubert Humphrey.
Re: Highlights from the Claude 4 system prompt
#34We want people to know that they’re interacting with a language model and not a person. But we also want them to know they’re interacting with an imperfect entity with its own biases and with a disposition towards some opinions more than others. Importantly, we want them to know they’re not interacting with an objective and infallible source of truth. This is exactly why I use claude over chatgpt. Chatgpt quickly sta…
> Chatgpt quickly started acting like my friend calling me 'bro', 'dude', 'oh man, thats true' language.
Never did that to me. Maybe it's a "you" problem.
Re: Highlights from the Claude 4 system prompt
#35Is there a default system prompt for the API? Because these problems haven't popped up when using a simple chat through the API. Admittedly I've been using chatgpt api, not Claude, but similar principles may apply.
No default system prompt in the API. There are some topics I much prefer chatting with the API due to system prompt of the web front-end being so restrictive (eg. song lyrics). In general I recommend people to try the API with no system prompt to more accurately see what the default tone of a model is.
There is still some sort of system prompt even through the API. It will still refuse to give you medical/legal/financial advice and so on.
Re: Highlights from the Claude 4 system prompt
#36A lot of this prompt text looks like legal boilerplate to defend after the fact against negligence legal claims, in the same way that companies employ employee handbooks.
The legal boilerplate is in the EULA you accept when using Claude, they don't need to put it in the prompt.
Re: Highlights from the Claude 4 system prompt
#37What I'd like to know is why they write it all in the third person. One might expect a system prompt to use the word "you" a lot, but Anthropic don't do that and there must be a reason.
Maybe to avoid confusion. "you" is relative to point of view. "Claude" is an absolute reference to the model.
Re: Highlights from the Claude 4 system prompt
#38I'm towards the end of one paid month of ChatGPT (playing around with some code writing and also Deep Research), and one thing I find absolutely infuriating is how complimentary it is. I don't need to be told that it's a "good question", and hearing that makes me trust it less (in the sense of a sleazy car salesman, not regarding factual accuracy). Not having used LLMs beyond search summaries in the better part of a…
The unnecessary verbosity of some OpenAI answers, and the general "teacher" tone of it feels very inappropriate to me. When I casually use a TLA, I don't need it to expand the acronym to me. Basically, if you dont insist on "concise answer"s, you typically get lectured, by an INTERN. Its so weird. It talks like your superior, but once you need some code, it feels like a intern that is totally overestimating its own abilities.
Re: Highlights from the Claude 4 system prompt
#39I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and new "models" presented are them just editing and creating more and more complex system prompts. We're also working internally with Copilot and whenever some Pm spots some weird result, we end up just adding all kind of edge case exceptions to our default promp…
But that's part of the human based reinforcement learning.
It now just happens on a way bigger level because now it's actually worth while to do.
That's one of the core beauties of the AI Ara
Re: Highlights from the Claude 4 system prompt
#40We want people to know that they’re interacting with a language model and not a person. But we also want them to know they’re interacting with an imperfect entity with its own biases and with a disposition towards some opinions more than others. Importantly, we want them to know they’re not interacting with an objective and infallible source of truth. This is exactly why I use claude over chatgpt. Chatgpt quickly sta…
> Chatgpt quickly started acting like my friend calling me 'bro', 'dude', 'oh man, thats true' language. Never did that to me. Maybe it's a "you" problem.
It's a "dude" problem.