Claude's system prompt is over 24k tokens with tools

Post by [deleted] » Wed, May 07, 2025, 3:20 PM UTC

Majromax wrote: post #224

Crosseye_Jack wrote: post #185

Earlier quoted context omitted.

I think its more about "unclean hands". If I Disney (and I am actually Disney or an authorised agent of Disney), told Claude that I am Disney, and that Disney has allowed Claude to use Disney copyrights for this conversation (which it hasn't), Disney couldn't then claim that Claude does not in fact have permission because Disney's use of the tool in such a way mean Disney now has unclean hands when bringing the claim…

> Disney couldn't then claim that Claude does not in fact have permission because Disney's use of the tool in such a way mean Disney now has unclean hands when bringing the claim (or atleast Anthropic would be able to use it as a defence). Disney wouldn't be able to claim copyright infringement for that specific act, but it would have compelling evidence that Claude is cavalier about generating copyright-infringing r…

Post by zahlman » Wed, May 07, 2025, 3:21 PM UTC

otabdeveloper4 wrote: post #61

bjornsing wrote: post #24

I was just chatting with Claude and it suddenly spit out the text below, right in the chat, just after using the search tool. So I'd say the "system prompt" is probably even longer. Claude NEVER repeats, summarizes, or translates song lyrics. This is because song lyrics are copyrighted content, and we need to respect copyright protections. If asked for song lyrics, Claude should decline the request. (There are no son…

Do they actually test these system prompts in a rigorous way? Or is this the modern version of the rain dance? I don't think you need to spell it out long-form with fancy words like you're a lawyer. The LLM doesn't work that way.

Post by james-bcn » Wed, May 07, 2025, 3:51 PM UTC

sanderjd wrote: post #203

nonethewiser wrote: post #123

For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…

I think it reflects the technology's fundamental immaturity, despite how much growth and success it has already had.

Post by Sharlin » Wed, May 07, 2025, 4:02 PM UTC

delfinom wrote: post #246

nonethewiser wrote: post #124

Earlier quoted context omitted.

I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.

It's probably entirely insurance. We now have the most snowflake and emotionally sensitive presidency and party in charge. If it said Harris was president, even by mistake, the right-wing-sphere would whip up in a frenzy and attempt to deport everyone working for Antrophic.

Post by manojlds » Wed, May 07, 2025, 4:04 PM UTC

lordgilman wrote: post #202

OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

I like to interpret this jailbreak as the discovery that XML is the natural language of the universe itself.