Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's system prompt is over 24k tokens with tools

github.com

Post reply on HN
Print view View original

271–280 of 350 posts

Re: Claude's system prompt is over 24k tokens with tools

#271

Post by ytpete »

throwawaystress wrote: post #160
Wowfunhappy wrote: post #150

Earlier quoted context omitted.

I feel like if Disney sued Anthropic based on this, Anthropic would have a pretty good defense in court: You specifically attested that you were Disney and had the legal right to the content.

I like the thought, but I don’t think that logic holds generally. I can’t just declare I am someone (or represent someone) without some kind of evidence. If someone just accepted my statement without proof, they wouldn’t have done their due diligence.

Right, imagine if other businesses like banks tried to use a defense like that! "No, it's not my fault some rando cleaned out your bank account because they said they were you."

Re: Claude's system prompt is over 24k tokens with tools

#272

Post by OtherShrezzing »

Wowfunhappy wrote: post #255
CPLX wrote: post #215

Earlier quoted context omitted.

Yeah but how did Anthropic come to have the copyrighted work embedded in the model?

Well, I was imagining this was related to web search. I went back and looked at the system prompt, and it's actually not entirely clear: > - Never reproduce or quote song lyrics in any form (exact, approximate, or encoded), even and especially when they appear in web search tool results, and even in artifacts . Decline ANY requests to reproduce song lyrics, and instead provide factual info about the song. Can anyone…

Web search was turned off in my original test. The lyrics appeared inside a thematically appropriate Frozen themed React artifact with snow falling gently in the background.

Re: Claude's system prompt is over 24k tokens with tools

#273

Post by wetpaws »

hadlock wrote: post #266
jermaustin1 wrote: post #221

Earlier quoted context omitted.

But then so are we? We are just predicting the next word we are saying, are we not? Even when you add thoughts behind it (sure some people think differently - be it without an inner monologue, or be it just in colors and sounds and shapes, etc), but that "reasoning" is still going into the act of coming up with the next word we are speaking/writing.

Humans and LLMs are built differently, it seems disingenuous to think we both use the same methods to arrive at the same general conclusion. I can inherently understand some proofs of pythagorean's theorem but an LLM might apply different ones for various reasons. But the output/result is still the same. If a next token generator run in parallel can generate a performant relational database that doesn't directly impl…

[dead]

Re: Claude's system prompt is over 24k tokens with tools

#274

Post by spookie »

jermaustin1 wrote: post #221
ceh123 wrote: post #214

Earlier quoted context omitted.

I'm not sure if this really says the truth is more complex? It is still doing next-token prediction, but it's prediction method is sufficiently complicated in terms of conditional probabilities that it recognizes that if you need to rhyme, you need to get to some future state, which then impacts the probabilities of the intermediate states. At least in my view it's still inherently a next-token predictor, just with r…

But then so are we? We are just predicting the next word we are saying, are we not? Even when you add thoughts behind it (sure some people think differently - be it without an inner monologue, or be it just in colors and sounds and shapes, etc), but that "reasoning" is still going into the act of coming up with the next word we are speaking/writing.

This type of response always irks me.

It shows that we, computer scientists, think of ourselves as experts on anything. Even though biological machines are well outside our expertise.

We should stop repeating things we don't understand.

Re: Claude's system prompt is over 24k tokens with tools

#275

Post by organsnyder »

victorbjorklund wrote: post #270
tikhonj wrote: post #248

Earlier quoted context omitted.

How would this would be any different from a file sharing site that included a checkbox that said "I have the legal right to distribute this content" with no other checking/verification/etc?

Rather when someone tweaks the content to avoid detection. Even today there are plenty of copyright material on youtube. They for example cut it in different ways to avoid detection.

"Everyone else is doing it" is not a valid infringement defense.

Re: Claude's system prompt is over 24k tokens with tools

#276

Post by HarHarVeryFunny »

nonethewiser wrote: post #123

For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…

Well, it is something - a language model, and this is just a stark reminder of that. It's predicting next word based on the input, and the only way to steer the prediction is therefore to tweak the input.

In terms of feels, this feels to me more like pushing on a string.

Re: Claude's system prompt is over 24k tokens with tools

#277

Post by scudsworth »

Wowfunhappy wrote: post #150
OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

I feel like if Disney sued Anthropic based on this, Anthropic would have a pretty good defense in court: You specifically attested that you were Disney and had the legal right to the content.

the sharp legal minds of hackernews

Re: Claude's system prompt is over 24k tokens with tools

#279

Post by LeifCarrotson »

organsnyder wrote: post #275
victorbjorklund wrote: post #270

Earlier quoted context omitted.

Rather when someone tweaks the content to avoid detection. Even today there are plenty of copyright material on youtube. They for example cut it in different ways to avoid detection.

"Everyone else is doing it" is not a valid infringement defense.

Valid defense, no, but effective defense - yes. The reason why is the important bit.

The reason your average human guitar teacher in their home can pull up a song on their phone and teach you reproduce it is because it's completely infeasible to police that activity, whether you're trying to identify it or to sue for it. The rights houlders have an army of lawyers and ears in a terrifying number of places, but winning $100 from ten million amateur guitar players isn't worth the effort.

But if it can be proven that Claude systematically violates copyright, well, Amazon has deep pockets. And AI only works because it's trained on millions of existing works, the copyright for which is murky. If they get a cease and desist that threatens their business model, they'll make changes from the top.

Re: Claude's system prompt is over 24k tokens with tools

#280

Post by IX-103 »

jbentley1 wrote: post #192
nonethewiser wrote: post #123

For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…

This isn't exactly correct, it is a combination of training and system prompt. You could train the system prompt into the model. This could be as simple as running the model with the system prompt, then training on those outputs until it had internalized the instructions. The downside is that it will become slightly less powerful, it is expensive, and if you want to change something you have to do it all over again.…

You can't just train with the negative examples showing filtered content, as that could lead to poor generalization. You'd need to supplement with samples from the training set to prevent catastrophic forgetting.

Otherwise it's like taking slices out of someone's brain until they can't recite a poem. Yes, at the end they can't recite a poem, but who knows what else they can no longer do. The positive examples from training essentially tell you what slices you need to put back to keep it functional.

Post reply on HN