[flagged]
Short AI. Hope you are more solvent than a glue factory!
Re. Prompt Length. Somewhere in the comments people talk about caching. Effectively it is zero cost.
Claude's system prompt is over 24k tokens with tools
161–170 of 350 posts
Re: Claude's system prompt is over 24k tokens with tools
#162Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…
Think of it like DRM: the point is not to make it completely impossible for anyone to ever break it. The point is to mitigate casual violations of policy.
Not that I like DRM! What I’m saying is that this is a business-level mitigation of a business-level harm, so jumping on the “it’s technically not perfect” angle is missing the point.
Re: Claude's system prompt is over 24k tokens with tools
#163Pretty cool.
However truly reliable, scalable LLM systems will need structured, modular architectures, not just brute-force long prompts. Think agent architectures with memory, state, and tool abstractions etc...not just bigger and bigger context windows.
Re: Claude's system prompt is over 24k tokens with tools
#164Pretty cool. However truly reliable, scalable LLM systems will need structured, modular architectures, not just brute-force long prompts. Think agent architectures with memory, state, and tool abstractions etc...not just bigger and bigger context windows.
[dead]
Re: Claude's system prompt is over 24k tokens with tools
#165Interestingly enough, sometimes "you" is used to give instructions (177 times), sometimes "Claude" (224 times). Is this just random based on who added the rule, or is there some purpose behind this differentiation?
There are a lot of inconsistencies like that. - (2 web_search and 1 web_fetch) - (3 web searches and 1 web fetch) - (5 web_search calls + web_fetch) which makes me wonder what's on purpose, empirical, or if they just let each team add something and collect some stats after a month.
I’ve noticed in my own prompt-writing that goes into code bases that it’s basically just programming, but… without any kind of consistency-checking, and with terrible refactoring tools. I find myself doing stuff like this all the time by accident.
One of many reasons I find the tech something to be avoided unless absolutely necessary.
Re: Claude's system prompt is over 24k tokens with tools
#166Earlier quoted context omitted.
I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.
Polls were all for Kamala except polymarket
Nonsense. Trump led in every swing state prior to election in aggregate poll analysis. Each swing state may have had an outlier Harris poll, but to say no polls existed with Trump leading is definitely incorrect. There were no surprise state outcomes at all in 2024, and the election was effectively over by 9pm Eastern time. Maybe you mean some kind of popular vote poll nationally, but that isnt how the US votes and also doesnt represent 'all polls'. I checked RCP archives and they show 7 polls for Harris leading nationally, and 10 polls for Harris losing nationally.
And let us not forget Harris was only even a candidate for 3 months. How Harris even makes it into the training window without Trump '24 result is already amazingly unlikely.
Re: Claude's system prompt is over 24k tokens with tools
#167Earlier quoted context omitted.
Do they actually test these system prompts in a rigorous way? Or is this the modern version of the rain dance? I don't think you need to spell it out long-form with fancy words like you're a lawyer. The LLM doesn't work that way.
They certainly do, and also offer the tooling to the public: https://docs.anthropic.com/en/docs/build-with-claude/prompt-... They also recommend to use it to iterate on your own prompts when using Claude Code for example
By "rigorous" I mean peeking under the curtain and actually quantifying the interactions between different system prompts and model weights.
"Chain of thought" and "reasoning" is marketing bullshit.
Re: Claude's system prompt is over 24k tokens with tools
#168Re: Claude's system prompt is over 24k tokens with tools
#169Re: Claude's system prompt is over 24k tokens with tools
#170Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…
It appears Microsoft Azure's content filtering policy prevents the prompt from being processed due to detecting the jailbreak, however, removing the tags and just leaving the text got me through with a successful response from GPT 4o.