Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

11–20 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#11

Post by daghamm »

fenomas wrote: post #7
daghamm wrote: post #6

These seem rather long. Do they count against my tokens for each conversation? One thing I have been missing in both chatgpt and Claude is the ability to exclude some part of the conversation or branch into two parts, in order to reduce the input size. Given how quickly they run out of steam, I think this could be an easy hack to improve performance and accuracy in long conversations.

I've wondered about this - you'd naively think it would be easy to run the model through the system prompt, then snapshot its state as of that point, and then handle user prompts starting from the cached state. But when I've looked at implementations it seems that's not done. Can anyone eli5 why?

My long dev session conversations are full of backtracking. This cannot be good for LLM performance.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#12

Post by pizza »

fenomas wrote: post #7
daghamm wrote: post #6

These seem rather long. Do they count against my tokens for each conversation? One thing I have been missing in both chatgpt and Claude is the ability to exclude some part of the conversation or branch into two parts, in order to reduce the input size. Given how quickly they run out of steam, I think this could be an easy hack to improve performance and accuracy in long conversations.

I've wondered about this - you'd naively think it would be easy to run the model through the system prompt, then snapshot its state as of that point, and then handle user prompts starting from the cached state. But when I've looked at implementations it seems that's not done. Can anyone eli5 why?

It def is done (kv caching the system prompt prefix) - they (Anthropic) also just released a feature that lets the end-user do the same thing to reduce in-cache token cost by 90% https://docs.anthropic.com/en/docs/build-with-claude/prompt-...

Re: Anthropic publishes the 'system prompts' that make Claude tick

#14

Post by hiddencost »

riku_iki wrote: post #5

its so long, so much waste of compute during inference. Wondering why they couldn't finetune it through some instructions.

Fine-tuning is expensive and slow compared to prompt engineering, for making changes to a production system.

You can develop validate and push a new prompt in hours.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#15

Post by cubefox »

atorodius wrote: post #3

Personally still amazed that we live in a time where we can tell a computer system in pure text how it should behave and it _kinda_ works

And "kinda" is an understatement. It understands you very well, perhaps even better than the average human would. (Average humans often don't understand jargon.)

Re: Anthropic publishes the 'system prompts' that make Claude tick

#17

Post by trevyn »

daghamm wrote: post #6

These seem rather long. Do they count against my tokens for each conversation? One thing I have been missing in both chatgpt and Claude is the ability to exclude some part of the conversation or branch into two parts, in order to reduce the input size. Given how quickly they run out of steam, I think this could be an easy hack to improve performance and accuracy in long conversations.

>Do they count against my tokens for each conversation?

This is for the Claude app, which is not billed in tokens, not the API.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#19

Post by mewpmewp2 »

tayo42 wrote: post #9

> whose only purpose is to fulfill the whims of its human conversation partners. > But of course that’s an illusion. If the prompts for Claude tell us anything, it’s that without human guidance and hand-holding, these models are frighteningly blank slates. Maybe more people should see what an llm is like without a stop token or trained to chat heh

It is like my mind right. It just goes on incessantly and uncontrollably without ever stopping.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#20

Post by WesolyKubeczek »

riku_iki wrote: post #5

its so long, so much waste of compute during inference. Wondering why they couldn't finetune it through some instructions.

I imagine the tone you set at the start affects the tone of responses, as it makes completions in that same tone more likely.

I would very much like to see my assumption checked — if you are as terse as possible in your system prompt, would it turn into a drill sergeant or an introvert?

Post reply on HN