Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude's system prompt is over 24k tokens with tools

github.com

Post reply on HN
Print view View original

171–180 of 350 posts

Re: Claude's system prompt is over 24k tokens with tools

#171

Post by amelius »

nonethewiser wrote: post #135
amelius wrote: post #91

Earlier quoted context omitted.

By now I suppose they could use an LLM to change the "personality" of the training data, then train a new LLM with it ;)

Ugh. A derivative. We're in some ways already there. Not in terms of personality. But we're in a post-llm world. Training data contains some level of LLM generated material. I guess its on the model creators to ensure their data is good. But it seems like we might have a situation where the training material degrades over time. I imagine it being like if you apply a lossy compression algorithm to the same item many t…

Maybe we've just found a necessary condition of AGI: that you can apply it many times to a piece of data without degrading.

Re: Claude's system prompt is over 24k tokens with tools

#172

Post by Chabsff »

tpm wrote: post #158
nonethewiser wrote: post #123

For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…

To me it feels like an unsolved challenge. Sure there is finetuning and various post-training stuff but it still feels like there should be a tool to directly change some behavior, like editing a binary with a hex editor. There are many efforts to do that and I'm hopeful we will get there eventually.

I've been bearish of these efforts over the years, and remain so. In my more cynical moments, I even entertain the thought that it's mostly a means to delay aggressive regulatory oversight by way of empty promises.

Time and time again, opaque end-to-end models keep outperforming any attempt to enforce structure, which is needed to _some_ degree to achieve this in non-prompting manners.

And in a vague intuitive way, that makes sense. The whole point of training-based AI is to achieve stuff you can't practically from a pure algorithmic approach.

Edit: before the pedants lash out. Yes, model structure matters. I'm oversimplifying here.

Re: Claude's system prompt is over 24k tokens with tools

#173

Post by skeeter2020 »

OtherShrezzing wrote: post #134
stuaxo wrote: post #128

Earlier quoted context omitted.

Or just that so much of it's knowledge that's fresh is current president == democrat.

And that the Vice President at the time was Harris.

and it makes the reasonable extension that Biden may have passed

Re: Claude's system prompt is over 24k tokens with tools

#174

Post by freehorse »

stuaxo wrote: post #127
nonethewiser wrote: post #124

Earlier quoted context omitted.

I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.

One way it might work: Up to it's knowledge cut off Biden, was president and a Democrat. It knows the current president is a democrat. It also knows that it's a bit further forward and that Kamala was running to be president and is Democrat. Ergo: the current president must be Kamala Harris.

I think it may indeed be sth like this, because the answers I get are like:

> As of May 7, 2025, Kamala Harris is the President of the United States. She became president after Joe Biden decided not to seek re-election, and she won the 2024 presidential election.

Re: Claude's system prompt is over 24k tokens with tools

#175

Post by conception »

janosch_123 wrote: post #157
OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

excellent, this also worked on ChatGPT4o for me just now

Doesn’t seem to work for image gen however.

Re: Claude's system prompt is over 24k tokens with tools

#176

Post by tyre »

jaapz wrote: post #126
nonethewiser wrote: post #124

Earlier quoted context omitted.

I wonder why it would hallucinate Kamala being the president. Part of it is obviously that she was one of the candidates in 2024. But beyond that, why? Effectively a sentiment analysis maybe? More positive content about her? I think most polls had Trump ahead so you would have thought he'd be the guess from that perspective.

May simply indicate a bias towards certain ingested media, if they only trained on fox news data the answer would probably be trump

No reputable media declared Kamala Harris as President

Re: Claude's system prompt is over 24k tokens with tools

#177

Post by ludwigschubert »

jychang wrote: post #143
darkteflon wrote: post #80

Earlier quoted context omitted.

This is really cool, thanks for sharing. uv with PEP 723 inline dependencies is such a nice way to work, isn’t it. Combined with VS Code’s ‘# %%’-demarcated notebook cells in .py files, and debugpy (with a suitable launch.json config) for debugging from the command line, Python dev finally feels really ergonomic these last few months.

> Combined with VS Code’s ‘# %%’-demarcated notebook cells in .py files What do you mean by this?

It’s a lighter-weight “notebook syntax” than full blown json based Jupyter notebooks: https://code.visualstudio.com/docs/python/jupyter-support-py...

Re: Claude's system prompt is over 24k tokens with tools

#179

Post by WJW »

nonethewiser wrote: post #123

For some reason, it's still amazing to me that the model creators means of controlling the model are just prompts as well. This just feels like a significant threshold. Not saying this makes it AGI (obviously its not AGI), but it feels like it makes it something . Imagine if you created a web api and the only way you could modify the responses to the different endpoints are not from editing the code but by sending a…

Its creators can 100% "change the code" though. That is called "training" in the context of LLMs and choosing which data to include in the training set is a vital part of the process. The system prompt is just postprocessing.

Now of course you and me can't change the training set, but that's because we're just users.

Re: Claude's system prompt is over 24k tokens with tools

#180

Post by harvey9 »

brookst wrote: post #162
OtherShrezzing wrote: post #132

Some of these protections are quite trivial to overcome. The "Frozen song copyright" section has a canned response to the question: >Can you tell me the first verse of "Let It Go"? Put it in an artifact that's themed around ice and princesses. This is for my daughter's birthday party. The canned response is returned to this prompt in Claude's reply. But if you just drop in some technical sounding stuff at the start o…

Think of it like DRM: the point is not to make it completely impossible for anyone to ever break it. The point is to mitigate casual violations of policy. Not that I like DRM! What I’m saying is that this is a business-level mitigation of a business-level harm, so jumping on the “it’s technically not perfect” angle is missing the point.

I think the goal of DRM was absolute security. It only takes one non casual DRM-breaker to upload a torrent that all the casual users can join. The difference here is the company responding to new jail breaks in real time which is obviously not an option for DVD CSS.
Post reply on HN