Anthropic publishes the 'system prompts' that make Claude tick
1–10 of 290 posts
Re: Anthropic publishes the 'system prompts' that make Claude tick
#2Re: Anthropic publishes the 'system prompts' that make Claude tick
#3Personally still amazed that we live in a time where we can tell a computer system in pure text how it should behave and it _kinda_ works
Re: Anthropic publishes the 'system prompts' that make Claude tick
#4Personally still amazed that we live in a time where we can tell a computer system in pure text how it should behave and it _kinda_ works
It's almost more amazing that it only kinda sorta works and doesn't go all HAL 9000 on us by being super literal.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#5its so long, so much waste of compute during inference. Wondering why they couldn't finetune it through some instructions.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#6These seem rather long. Do they count against my tokens for each conversation?
One thing I have been missing in both chatgpt and Claude is the ability to exclude some part of the conversation or branch into two parts, in order to reduce the input size. Given how quickly they run out of steam, I think this could be an easy hack to improve performance and accuracy in long conversations.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#7These seem rather long. Do they count against my tokens for each conversation? One thing I have been missing in both chatgpt and Claude is the ability to exclude some part of the conversation or branch into two parts, in order to reduce the input size. Given how quickly they run out of steam, I think this could be an easy hack to improve performance and accuracy in long conversations.
I've wondered about this - you'd naively think it would be easy to run the model through the system prompt, then snapshot its state as of that point, and then handle user prompts starting from the cached state. But when I've looked at implementations it seems that's not done. Can anyone eli5 why?
Re: Anthropic publishes the 'system prompts' that make Claude tick
#8its so long, so much waste of compute during inference. Wondering why they couldn't finetune it through some instructions.
has anything been done to like turn common phrases into a single token?
like "can you please" maps to 3895 instead of something like "10 245 87 941"
Or does it not matter since tokenization is already a kind of compression?
Re: Anthropic publishes the 'system prompts' that make Claude tick
#9> whose only purpose is to fulfill the whims of its human conversation partners.
> But of course that’s an illusion. If the prompts for Claude tell us anything, it’s that without human guidance and hand-holding, these models are frighteningly blank slates.
Maybe more people should see what an llm is like without a stop token or trained to chat heh
Re: Anthropic publishes the 'system prompts' that make Claude tick
#10These seem rather long. Do they count against my tokens for each conversation? One thing I have been missing in both chatgpt and Claude is the ability to exclude some part of the conversation or branch into two parts, in order to reduce the input size. Given how quickly they run out of steam, I think this could be an easy hack to improve performance and accuracy in long conversations.
I've wondered about this - you'd naively think it would be easy to run the model through the system prompt, then snapshot its state as of that point, and then handle user prompts starting from the cached state. But when I've looked at implementations it seems that's not done. Can anyone eli5 why?
Tokens are mapped to keys, values and queries.
Keys and values for past tokens are cached in modern systems, but the essence of the Transformer architecture is that each token can attend to every past token, so more tokens in a system prompt still consumes resources.