Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Highlights from the Claude 4 system prompt

simonwillison.net

Post reply on HN
Print view View original

41–50 of 95 posts

Re: Highlights from the Claude 4 system prompt

#41

Post by antupis »

mns wrote: post #31

I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and new "models" presented are them just editing and creating more and more complex system prompts. We're also working internally with Copilot and whenever some Pm spots some weird result, we end up just adding all kind of edge case exceptions to our default promp…

I think we already hit somekind of performance wall begin of this year. It feels that models are now balancing between rule following and agentic case and general stuff. eg Claude 4 sonet just feels better in Cursor and follows rules very well, and same time it gets equal or worse scores in benchmark against 3.7 Sonet.

Re: Highlights from the Claude 4 system prompt

#42

Post by svantana »

cma wrote: post #33
svantana wrote: post #32

Earlier quoted context omitted.

> the outcome of the 46th presidential election By my count, the winner of the 46th US presidential election was Nixon. I would be pretty surprised if any chatbot managed to get that right.

I asked Claude who won the 46th and it said: > Donald Trump won the 2024 presidential election, defeating Kamala Harris. He was inaugurated as the 47th president on January 20, 2025. > Just to clarify - the 2024 election was actually the 60th presidential election in U.S. history, not the 46th. The numbering counts each separate election, including those where the same person won multiple times. It got a followup wro…

I'm not surprised it works with that amount of hand-holding.

Re: Highlights from the Claude 4 system prompt

#43

Post by 0xTJ »

addaon wrote: post #14
0xTJ wrote: post #9

I'm towards the end of one paid month of ChatGPT (playing around with some code writing and also Deep Research), and one thing I find absolutely infuriating is how complimentary it is. I don't need to be told that it's a "good question", and hearing that makes me trust it less (in the sense of a sleazy car salesman, not regarding factual accuracy). Not having used LLMs beyond search summaries in the better part of a…

> hearing that makes me trust it less That seems like a good thing, given that... > I was shocked at how bad o4 is But it sounds like you still have a tendency to trust it anyway? Anything that they can do to make it seem less trustworthy -- and it seems pretty carefully tuned right now to generate text that reminds humans of a caricature of a brown-nosing nine year old rather than another human -- is probably a net…

As I say in the parentheses after that first comment on trusting it less, I mean it in a human sense, a person being two-faced or ready to exploit you then stab you in the back. While that and factual inaccuracy aren't mutually exclusive, the first paragraph is about how the tone used makes it seem personally untrustworthy.

Being surprised at how poorly it did doesn't mean that I trusted the results in the first place. I had just expected it not to fail so spectacularly and confidently at this point in development.

Also, myself interpreting that tone as untrustworthy (again, in the sense of personality, not information) doesn't mean that others will perceive it in the same way. I was going into this with knowledge of the specific field and an expectation for precise, accurate, and concise communication. I would equally mistrust a human using so much "fluff" and giving such unearned and unnecessary compliments.

Re: Highlights from the Claude 4 system prompt

#45

Post by therealpygon »

handfuloflight wrote: post #7

Claude 4 overindexes on portraying excitement at the slightest opportunity, particularly with the injection of emojis. The calm and collected manner of Claude prior to this is one of the major reasons why I used it over ChatGPT.

Well, they gotta cater to the LM Arena scoring of Gen whatever and internet degenerates that both want emojis in their text.

Re: Highlights from the Claude 4 system prompt

#48

Post by pegasus »

dist-epoch wrote: post #35
Stagnant wrote: post #30

Earlier quoted context omitted.

No default system prompt in the API. There are some topics I much prefer chatting with the API due to system prompt of the web front-end being so restrictive (eg. song lyrics). In general I recommend people to try the API with no system prompt to more accurately see what the default tone of a model is.

There is still some sort of system prompt even through the API. It will still refuse to give you medical/legal/financial advice and so on.

That behavior is baked in the model via RLHF, it doesn't require a specific prompt.

Re: Highlights from the Claude 4 system prompt

#49

Post by merksittich »

mns wrote: post #31

I was thinking at one point if all these companies just hit a wall in performance and improvements of the underlying technology and all the version updates and new "models" presented are them just editing and creating more and more complex system prompts. We're also working internally with Copilot and whenever some Pm spots some weird result, we end up just adding all kind of edge case exceptions to our default promp…

Speaking of performance wall: The Claude 4 results were added to the Aider LLM Leaderboard [0] yesterday. Opus 4 is clearly below Gemini 2.5 Pro at almost twice the price. Sonnet 4 fares worse than Sonnet 3.7, with the thinking version of Sonnet 4 being somewhat cheaper than its 3.7 counterpart.

[0] https://aider.chat/docs/leaderboards/

Re: Highlights from the Claude 4 system prompt

#50

Post by crmd »

dist-epoch wrote: post #36
crmd wrote: post #16

A lot of this prompt text looks like legal boilerplate to defend after the fact against negligence legal claims, in the same way that companies employ employee handbooks.

The legal boilerplate is in the EULA you accept when using Claude, they don't need to put it in the prompt.

An EULA and an Employee Handbook serve different legal purposes.

The reason handbooks for example say “downloading or transmitting copyrighted material on the company network is strictly prohibited”, is so if a copyright holder attempts to sue the company for an employee’s illegal actions, it can prove it had taken reasonable steps during training to inform employees that the action was strictly prohibited, and their asses are therefore covered.

I’m speculating that the system prompt may serve a similar legal function: even if the LLM did transmit copyrighted song lyrics, we are not liable because as you can see right here in the system prompt we told it not to do that.

Post reply on HN