Live data from Hacker News

Claude's system prompt is over 24k tokens with tools

github.com

11–20 of 350 posts

Re: Claude's system prompt is over 24k tokens with tools

#11
post #10

I like how there are IFs and ELSE IFs but those logical constructs aren't actually explicitly followed... and inside the IF instead of a dash as a bullet point there's an arrow.. that's the _syntax_? hah.. what if there were two lines of instructions, you'd make a new line starting with another arrow..? Did they try some form of it without IFs first?...

[deleted]

Re: Claude's system prompt is over 24k tokens with tools

#13
post #10

I like how there are IFs and ELSE IFs but those logical constructs aren't actually explicitly followed... and inside the IF instead of a dash as a bullet point there's an arrow.. that's the _syntax_? hah.. what if there were two lines of instructions, you'd make a new line starting with another arrow..? Did they try some form of it without IFs first?...

Can you guess who wrote that ?

Re: Claude's system prompt is over 24k tokens with tools

#14
In addition to having long system prompts, you also need to provide agents with the right composable tools to make it work.

I’m having reasonable success with these seven tools: read, write, diff, browse, command, ask, think.

There is a minimal template here if anyone finds it useful: https://github.com/aperoc/toolkami

Re: Claude's system prompt is over 24k tokens with tools

#15
post #10

I like how there are IFs and ELSE IFs but those logical constructs aren't actually explicitly followed... and inside the IF instead of a dash as a bullet point there's an arrow.. that's the _syntax_? hah.. what if there were two lines of instructions, you'd make a new line starting with another arrow..? Did they try some form of it without IFs first?...

[deleted]

Re: Claude's system prompt is over 24k tokens with tools

#16
How did they leak it, jailbreak? Was this confirmed? I am checking for the situation where the true instructions are not what is being reported here. The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal.

Re: Claude's system prompt is over 24k tokens with tools

#17

How did they leak it, jailbreak? Was this confirmed? I am checking for the situation where the true instructions are not what is being reported here. The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal.

All System Prompts from Anthropic models are public information, released by Anthropic themselves: https://docs.anthropic.com/en/release-notes/system-prompts. I'm unsure (I just skimmed through) to what the differences between this and the publicly released ones are, so they're might be some differences.

Re: Claude's system prompt is over 24k tokens with tools

#18

How did they leak it, jailbreak? Was this confirmed? I am checking for the situation where the true instructions are not what is being reported here. The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal.

> The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal.

How would you detect this? I always wonder about this when I see a 'jail break' or similar for LLM...

Re: Claude's system prompt is over 24k tokens with tools

#19

How did they leak it, jailbreak? Was this confirmed? I am checking for the situation where the true instructions are not what is being reported here. The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal.

> The language model could have "hallucinated" its own system prompt instructions, leaving no guarantee that this is the real deal. How would you detect this? I always wonder about this when I see a 'jail break' or similar for LLM...

In this case it’s easy: get the model to output its own system prompt and then compare to the published (authoritative) version.

The actual system prompt, the “public” version, and whatever the model outputs could all be fairly different from each other though.

Post reply on HN