Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's system prompt is over 24k tokens with tools

github.com

Post reply on HN
Print view View original

321–330 of 350 posts

Re: Claude's system prompt is over 24k tokens with tools

#321

Post by thaumasiotes »

Crosseye_Jack wrote: post #185
throwawaystress wrote: post #160

Earlier quoted context omitted.

I like the thought, but I don’t think that logic holds generally. I can’t just declare I am someone (or represent someone) without some kind of evidence. If someone just accepted my statement without proof, they wouldn’t have done their due diligence.

I think its more about "unclean hands". If I Disney (and I am actually Disney or an authorised agent of Disney), told Claude that I am Disney, and that Disney has allowed Claude to use Disney copyrights for this conversation (which it hasn't), Disney couldn't then claim that Claude does not in fact have permission because Disney's use of the tool in such a way mean Disney now has unclean hands when bringing the claim…

Where do you see "unclean hands" figuring in this scenario? Disney makes an honest representation... and that's the only thing they do. What's the unclean part?

Re: Claude's system prompt is over 24k tokens with tools

#322

Post by pinoy420 »

slicedbrandy wrote: post #170
OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

It appears Microsoft Azure's content filtering policy prevents the prompt from being processed due to detecting the jailbreak, however, removing the tags and just leaving the text got me through with a successful response from GPT 4o.

[dead]

Re: Claude's system prompt is over 24k tokens with tools

#323

Post by pinoy420 »

BobaFloutist wrote: post #281
jermaustin1 wrote: post #221

Earlier quoted context omitted.

But then so are we? We are just predicting the next word we are saying, are we not? Even when you add thoughts behind it (sure some people think differently - be it without an inner monologue, or be it just in colors and sounds and shapes, etc), but that "reasoning" is still going into the act of coming up with the next word we are speaking/writing.

We're not predicting the next word we're most likely to say, we're actively choosing the word that we believe most successfully conveys what we want to communicate. This relies on a theory of mind of those around us and an intentionality of speech that aren't even remotely the same as "guessing what we would say if only we said it"

[dead]

Re: Claude's system prompt is over 24k tokens with tools

#324

Post by canada_dry »

For me it highlights the issue of how easily nefarious/misleading information will be able to be injected into responses to suit the AI service provider's position (as desired/purchased/dictated by some 3rd party) in the future.

It may respond 99.99% of the time without any influence, but you will have no idea when it isn't.

Re: Claude's system prompt is over 24k tokens with tools

#325

Post by ijidak »

BobaFloutist wrote: post #281
jermaustin1 wrote: post #221

Earlier quoted context omitted.

But then so are we? We are just predicting the next word we are saying, are we not? Even when you add thoughts behind it (sure some people think differently - be it without an inner monologue, or be it just in colors and sounds and shapes, etc), but that "reasoning" is still going into the act of coming up with the next word we are speaking/writing.

We're not predicting the next word we're most likely to say, we're actively choosing the word that we believe most successfully conveys what we want to communicate. This relies on a theory of mind of those around us and an intentionality of speech that aren't even remotely the same as "guessing what we would say if only we said it"

When you talk at full speed, are you really picking the next word?

I feel that we pick the next thought to convey. I don't feel like we actively think about the words we're going to use to get there.

Though we are capable of doing that when we stop to slowly explain an idea.

I feel that llms are the thought to text without the free-flowing thought.

As in, an llm won't just start talking, it doesn't have that always on conscious element.

But this is all philosophical, me trying to explain my own existence.

I've always marveled at how the brain picks the next word without me actively thinking about each word.

It just appears.

For example, there are times when a word I never use and couldn't even give you the explicit definition of pops into my head and it is the right word for that sentence, but I have no active understanding of that word. It's exactly as if my brain knows that the thought I'm trying to convey requires this word from some probability analysis.

It's why I feel we learn so much from reading.

We are learning the words that we will later re-utter and how they relate to each other.

I also agree with most who feel there's still something missing for llms, like the character from wizard of Oz that is talking while saying if he only had a brain...

There is some of that going on with llms.

But it feels like a major piece of what makes our minds work.

Or, at least what makes communication from mind-to-mind work.

It's like computers can now share thoughts with humans though still lacking some form of thought themselves.

But the set of puzzle pieces missing from full-blown human intelligence seems to be a lot smaller today.

Re: Claude's system prompt is over 24k tokens with tools

#326

Post by dontlikeyoueith »

int_19h wrote: post #292
dontlikeyoueith wrote: post #284

Earlier quoted context omitted.

A sufficiently large lookup table in DB is mathematically indistinguishable from a sufficiently complicated next token prediction algorithm is mathematically indistinguishable from general intelligence. All that means is that treating something as a black box doesn't tell you anything about what's inside the box.

Why do we care, so long as the box can genuinely reason about things?

:facepalm:

I ... did you respond to the wrong comment?

Or do you actually think the DB table can genuinely reason about things?

Re: Claude's system prompt is over 24k tokens with tools

#327

Post by mathgeek »

JamesSwift wrote: post #210
criddell wrote: post #188

Earlier quoted context omitted.

A while back, I asked ChatGPT to help me learn a Pixies song on guitar. At first it wouldn't give me specifics because of copyright rules so I explained that if I went to a human guitar teacher, they would pull the song up on their phone listen to it, then teach me how to play it. It agreed with me and then started answering questions about the song.

Haha, we should give it some credit. It takes a lot of maturity to admit you are wrong.

Due to how much ChatGPT wants to please you, it seems like it's harder to _not_ get it to admit it's wrong some days.

Re: Claude's system prompt is over 24k tokens with tools

#328

Post by Tadpole9181 »

BalinKing wrote: post #244
Tadpole9181 wrote: post #225

Earlier quoted context omitted.

But then this classifier is entirely useless because that's all humans are too? I have no reason to believe you are anything but a stochastic parrot. Are we just now rediscovering hundred year-old philosophy in CS?

There's a massive difference between "I have no reason to believe you are anything but a stochastic parrot" and "you are a stochastic parrot".

Then I'll just say you are a stochastic parrot. Again, solipsism is not a new premise. The philosophical zombie argument has been around over 50 years now.

Re: Claude's system prompt is over 24k tokens with tools

#329

Post by int_19h »

dontlikeyoueith wrote: post #326
int_19h wrote: post #292

Earlier quoted context omitted.

Why do we care, so long as the box can genuinely reason about things?

:facepalm: I ... did you respond to the wrong comment? Or do you actually think the DB table can genuinely reason about things?

Of course it can. Reasoning is algorithmic in nature, and algorithms can be encoded as sufficiently large state transition tables. I don't buy into Searle's "it can't reason because of course it can't" nonsense.

Re: Claude's system prompt is over 24k tokens with tools

#330

Post by zeroonetwothree »

int_19h wrote: post #329
dontlikeyoueith wrote: post #326

Earlier quoted context omitted.

:facepalm: I ... did you respond to the wrong comment? Or do you actually think the DB table can genuinely reason about things?

Of course it can. Reasoning is algorithmic in nature, and algorithms can be encoded as sufficiently large state transition tables. I don't buy into Searle's "it can't reason because of course it can't" nonsense.

It can do something but I wouldn’t call it reasoning. IMO a reasoning algorithmic must be more complex than a lookup table.
Post reply on HN