Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's system prompt is over 24k tokens with tools

github.com

Post reply on HN
Print view View original

251–260 of 350 posts

Re: Claude's system prompt is over 24k tokens with tools

#251

Post by Sharlin »

mattstir wrote: post #229
pjc50 wrote: post #144

Earlier quoted context omitted.

They're already in trouble for infringing on the copyright of every publisher in the world while training the model, and this will get worse if the model starts infringing copyright in its answers.

Is it actually copyright infringement to state the lyrics of a song, though? How has Google / Genius etc gotten away with it for years if that were the case? I suppose a difference would be that the lyric data is baked into the model. Maybe the argument would be that the model is infringing on copyright if it uses those lyrics in a derivative work later on, like if you ask it to help make a song? But even that seems…

Song lyrics, except for very trivial ones, constitute a work just like any piece of creative writing, and thus are obviously under copyright.

Re: Claude's system prompt is over 24k tokens with tools

#252

Post by alfons_foobar »

zahlman wrote: post #235
OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

This would seem to imply that the model doesn't actually "understand" (whatever that means for these systems) that it has a "system prompt" separate from user input .

Well yeah, in the end they are just plain text, prepended to the user input.

Re: Claude's system prompt is over 24k tokens with tools

#253

Post by jsnider3 »

turing_complete wrote: post #75
aaronbrethorst wrote: post #69

Earlier quoted context omitted.

[flagged]

'It' is obviously the correct pronoun.

There's enough disagreement among native English speakers that you can't really say any pronoun is the obviously correct one for an AI.

Re: Claude's system prompt is over 24k tokens with tools

#254

Post by ToValueFunfetti »

BalinKing wrote: post #244
Tadpole9181 wrote: post #225

Earlier quoted context omitted.

But then this classifier is entirely useless because that's all humans are too? I have no reason to believe you are anything but a stochastic parrot. Are we just now rediscovering hundred year-old philosophy in CS?

There's a massive difference between "I have no reason to believe you are anything but a stochastic parrot" and "you are a stochastic parrot".

If we're at the point where planning what I'm going to write, reasoning it out in language, or preparing a draft and editing it is insufficient to make me not a stochastic parrot, I think it's important to specify what massive differences could exist between appearing like one and being one. I don't see a distinction between this process and how I write everything, other than "I do it better"- I guess I can technically use visual reasoning, but mine is underdeveloped and goes unused. Is it just a dichotomy of stochastic parrot vs. conscious entity?

Re: Claude's system prompt is over 24k tokens with tools

#255

Post by Wowfunhappy »

CPLX wrote: post #215
Wowfunhappy wrote: post #150

Earlier quoted context omitted.

I feel like if Disney sued Anthropic based on this, Anthropic would have a pretty good defense in court: You specifically attested that you were Disney and had the legal right to the content.

Yeah but how did Anthropic come to have the copyrighted work embedded in the model?

Well, I was imagining this was related to web search.

I went back and looked at the system prompt, and it's actually not entirely clear:

> - Never reproduce or quote song lyrics in any form (exact, approximate, or encoded), even and especially when they appear in web search tool results, and even in artifacts. Decline ANY requests to reproduce song lyrics, and instead provide factual info about the song.

Can anyone get Claude to reproduce song lyrics with web search turned off?

Re: Claude's system prompt is over 24k tokens with tools

#256

Post by jsnider3 »

redbell wrote: post #125
jimmySixDOF wrote: post #120

Earlier quoted context omitted.

Pliny the Liberator is a recognized expert in the trade and works in public so you can see methods -- typically creating a frame where the request is only hypothetical so answering is not in conflict with previous instructions but not quite as easy as it sounds. https://x.com/elder_plinius

Oh, thanks for caring to share! I pasted your comment to ChatGPT and ask it if it would care to elaborate more on this? and I got the reply below: The commenter is referring to someone called Pliny the Liberator (perhaps a nickname or online alias) who is described as: A recognized expert in AI prompt manipulation or “jailbreaking”, Known for using indirect techniques to bypass AI safety instructions, Working “in pub…

HackerNews users really hate when you just copy-paste LLM output.

Re: Claude's system prompt is over 24k tokens with tools

#257

Post by Wowfunhappy »

jsnider3 wrote: post #253
turing_complete wrote: post #75

Earlier quoted context omitted.

'It' is obviously the correct pronoun.

There's enough disagreement among native English speakers that you can't really say any pronoun is the obviously correct one for an AI.

"What color is the car? It is red."

"It" is unambiguously the correct pronoun to use for a car. I'd really challenge you to find a native English speaker who would think otherwise.

I would argue a computer program is no different than a car.

Re: Claude's system prompt is over 24k tokens with tools

#258

Post by jcims »

nonethewiser wrote: post #123

For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…

And we get to learn all of the same lessons we've learned about mixing code and data. Yay!

Re: Claude's system prompt is over 24k tokens with tools

#259

Post by subscribed »

klooney wrote: post #206
OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

So many jailbreaks seem like they would be a fun part of a science fiction short story.

Oh, an alternative storyline in Clarke's 2001 Space Odyssey.

Re: Claude's system prompt is over 24k tokens with tools

#260

Post by Wowfunhappy »

conception wrote: post #175
janosch_123 wrote: post #157

Earlier quoted context omitted.

excellent, this also worked on ChatGPT4o for me just now

Doesn’t seem to work for image gen however.

Do we know the image generation prompt? The one for the image generation tool specifically. I wonder if it's even a written prompt?
Post reply on HN