Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's system prompt is over 24k tokens with tools

github.com

Post reply on HN
Print view View original

111–120 of 350 posts

Re: Claude's system prompt is over 24k tokens with tools

#111

Post by kikimora »

vidarh wrote: post #97
mrweasel wrote: post #72

Earlier quoted context omitted.

I don't know if I feel cheated, but it seems a little unmanageable. How is this suppose to scale? How the hell do you even start to debug the LLM when it does something incorrect? It's not like you can attach a debugger to English. The "vibe" I'm getting is that of a junior developer who slows problems be tacking on an ever increasing amount of code, rather than going back an fixing underlying design flaws.

See it as a temporary workaround, and assume each instruction will also lead to additional training data to try to achieve the same in the next model directly.

It comes down to solving this - given instruction X find out how to change the training data such that X is obeyed and none other side effects appears. Given amount if the training data and complexities of involved in training I don’t think there is a clear way to do it.

Re: Claude's system prompt is over 24k tokens with tools

#112

Post by mcintyre1994 »

sramam wrote: post #5

do tools like cursor get a special pass? Or do they do some magic? I'm always amazed at how well they deal with diffs. especially when the response jank clearly points to a "... + a change", and cursor maps it back to a proper diff.

I think Cursor would need to have their own system prompt for most of this, I don't think the API includes much of this.

Re: Claude's system prompt is over 24k tokens with tools

#113

Post by mcintyre1994 »

beardedwizard wrote: post #64
moralestapia wrote: post #59

Earlier quoted context omitted.

>What makes you think the others are significantly different? ChatGPT's prompt is on the order of 1k, if the leaks turn out to be real. Even that one seems a bit high for my taste, but they're the experts, not me. >It’s logical to use the context to tell it what to do. You probably don't know much about this, but no worries I can explain. You can train a model to "become" anything you want, if your default prompt sta…

But this is the solution the most cutting edge llm research has yielded, how do you explain that? Are they just willfully ignorant at OpenAI and anthropic? If fine tuning is the answer why aren't the best doing it?

I'd guess the benefit is that it's quicker/easier to experiment with the prompt? Claude has prompt caching, I'm not sure how efficient that is but they offer a discount on requests that make use of it. So it might be that that's efficient enough that it's worth the tradeoff for them?

Also I don't think much of this prompt is used in the API, and a bunch of it is enabling specific UI features like Artifacts. So if they re-use the same model for the API (I'm guessing they do but I don't know) then I guess they're limited in terms of fine tuning.

Re: Claude's system prompt is over 24k tokens with tools

#114

Post by FooBarWidget »

azinman2 wrote: post #60
FooBarWidget wrote: post #49

Earlier quoted context omitted.

The other day I was talking to Grok, and then suddenly it started outputting corrupt tokens, after which it outputted the entire system prompt. I didn't ask for it. There truly are a million ways for LLMs to leak their system prompt.

What did it say?

I didn't save the conversation but one of the things that stood out was a long list of bullets saying that Grok doesn't know anything about x/AI pricing or product details, tell user to go x/AI website rather than making things up. This section seems to be longer than the section that defines what Grok is.

Nothing about tool calling.

Re: Claude's system prompt is over 24k tokens with tools

#115

Post by redbell »

I believe tricking a system to reveal its system prompt is the new reverse engineering, and I've been wondering what techniques are used to extract this type of information?

For instance, major AI-powered IDEs had their system prompts revealed and published publicly: https://github.com/x1xhlol/system-prompts-and-models-of-ai-t...

Re: Claude's system prompt is over 24k tokens with tools

#116

Post by freehorse »

late2part wrote: post #110
freehorse wrote: post #98

Earlier quoted context omitted.

I asked it this exact question, to anybody curious https://claude.ai/share/ea4aa490-e29e-45a1-b157-9acf56eb7f8a edit:fixed link

The conversation you were looking for could not be found.

oops, fixed

Re: Claude's system prompt is over 24k tokens with tools

#118

Post by mkl »

faustocarva wrote: post #107
freehorse wrote: post #96

Earlier quoted context omitted.

This system prompt is not used in the API, so it is not relevant for the perceived personality of the model if you do not use it through claude.ai interface, eg through an editor etc.

Why this? Because for OpenAI you can set it using API.

I think you misread. With the API you're not using this standard chat system prompt, but whatever one you set: https://docs.anthropic.com/en/docs/build-with-claude/prompt-...

Re: Claude's system prompt is over 24k tokens with tools

#119

Post by vidarh »

kikimora wrote: post #111
vidarh wrote: post #97

Earlier quoted context omitted.

See it as a temporary workaround, and assume each instruction will also lead to additional training data to try to achieve the same in the next model directly.

It comes down to solving this - given instruction X find out how to change the training data such that X is obeyed and none other side effects appears. Given amount if the training data and complexities of involved in training I don’t think there is a clear way to do it.

I'm slightly less sceptical that they can do it, but we presumably agree that changing the prompt is far faster, and so you change the prompt first, and the prompt effectively will serve in part as documentation of issues to chip away at while working on the next iterations of the underlying models.

Re: Claude's system prompt is over 24k tokens with tools

#120

Post by jimmySixDOF »

redbell wrote: post #115

I believe tricking a system to reveal its system prompt is the new reverse engineering , and I've been wondering what techniques are used to extract this type of information? For instance, major AI-powered IDEs had their system prompts revealed and published publicly: https://github.com/x1xhlol/system-prompts-and-models-of-ai-t...

Pliny the Liberator is a recognized expert in the trade and works in public so you can see methods -- typically creating a frame where the request is only hypothetical so answering is not in conflict with previous instructions but not quite as easy as it sounds.

https://x.com/elder_plinius

Post reply on HN