Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

171–180 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#171

Post by bhelkey »

ForHackernews wrote: post #166

"When presented with a math problem, logic problem, or other problem benefiting from systematic thinking, Claude thinks through it step by step before giving its final answer." ... do AI makers believe this works? Like do think Claude is a conscious thing that can be instructed to "think through" a problem? All of these prompts (from Anthropic and elsewhere) have a weird level of anthropomorphizing going on. Are AI c…

LLMs predict the next token. Imagine someone said to you, "it takes a musician 10 minutes to play a song, how long will it take for 5 musicians to play? I will work through the problem step by step".

What are they more likely to say next? The reasoning behind their answer? Or a number of minutes?

People rarely say, "let me describe my reasoning step by step. The answer is 10 minutes".

Re: Anthropic publishes the 'system prompts' that make Claude tick

#172

Post by brulard »

armchairhacker wrote: post #167
creatonez wrote: post #134

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…

How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.

I believe Claude is aware if information close to the one retrieved from the vector space is scarce. I'm no expert, but i imagine it makes a query to the vector database and get the data close enough to places pointed out by the prompt. And it may see that part of the space is quite empty. If this is far off, someone please explain.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#173

Post by cjbillington »

ForHackernews wrote: post #166

"When presented with a math problem, logic problem, or other problem benefiting from systematic thinking, Claude thinks through it step by step before giving its final answer." ... do AI makers believe this works? Like do think Claude is a conscious thing that can be instructed to "think through" a problem? All of these prompts (from Anthropic and elsewhere) have a weird level of anthropomorphizing going on. Are AI c…

They believe it works because it does work!

"Chain of thought" prompting is a well-established method to get better output from LLMs.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#174

Post by lemming »

xienze wrote: post #155
creatonez wrote: post #134

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…

> Probably for the best that users see the words "Sorry, I hallucinated" every now and then. Wouldn’t “sorry, I don’t know how to answer the question” be better?

"Sorry, I just made that up" is more accurate.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#175

Post by blooalien »

marcus0x62 wrote: post #120
gwervc wrote: post #80

Earlier quoted context omitted.

How do glorified Markov chains think?

I understand it to be by predicting the next most likely output token based on previous user input. I also understand that, simplistic though the above explanation is and perhaps is even wrong in some way, it to be a more thorough explanation than anyone thus far has been able to provide about how, exactly, human consciousness and thought works. In any case, my point is this: nobody can say “LLMs don’t reason in the…

Does your brain completely stop doing anything between verbal statements (output)? An LLM does stop doing stuff between requests to generate a string of language tokens (their entire purpose). When not actually generating tokens, an LLM doesn't sit there and think things like "Was what I just said correct?" or "Hmm. That was an interesting discussion. I think I'll go research more on the topic". Nope. It just sits there idle, waiting for another request to generate text. Does your brain ever sit 100% completely idle?

Re: Anthropic publishes the 'system prompts' that make Claude tick

#176

Post by th0ma5 »

brulard wrote: post #172
armchairhacker wrote: post #167

Earlier quoted context omitted.

How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.

I believe Claude is aware if information close to the one retrieved from the vector space is scarce. I'm no expert, but i imagine it makes a query to the vector database and get the data close enough to places pointed out by the prompt. And it may see that part of the space is quite empty. If this is far off, someone please explain.

I think good, true, but rare information would also fit that definition so it'd be a shame if it discovered something that could save humanity but then discounted it as probably not accurate.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#177

Post by SparkyMcUnicorn »

generalizations wrote: post #123

Claude has been pretty great. I stood up an 'auto-script-writer' recently, that iteratively sends a python script + prompt + test results to either GPT4 or Claude, takes the output as a script, runs tests on that, and sends those results back for another loop. (Usually took about 10-20 loops to get it right) After "writing" about 5-6 python scripts this way, it became pretty clear that Claude is far, far better - if…

My experience reflects this, generally speaking.

I've found that GPT-4o is better than Sonnet 3.5 at writing in certain languages like rust, but maybe that's just because I'm better at prompting openai models.

Latest example I recently ran was a rust task that went 20 loops without getting a successful compile in sonnet 3.5, but compiled and was correct with gpt-4o on the second loop.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#178

Post by AcerbicZero »

My big complaint with claude is that it burns up all its credits as fast as possible and then gives up; We'll get about half way through a problem and claude will be trying to rewrite its not very good code for the 8th time without being asked and next thing I know I'm being told I have 3 messages left.

Pretty much insta cancelled my subscription. If I was throwing a few hundred API calls at it, every min, ok, sure, do what you gotta do, but the fact that I can burn out the AI credits just by typing a few questions over the course of half a morning is just sad.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#179

Post by lemming »

Nihilartikel wrote: post #150
digging wrote: post #106

Earlier quoted context omitted.

Odd how many of those instructions are almost always ignored (eg. "don't apologize," "don't explain code without being asked"). What is even the point of these system prompts if they're so weak?

Same with my kindergartener! Like, what's their use if I have to phrase everything as an imperative command?

Much like the LLMs, in a few years their capabilities will be much improved and you won't have to.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#180

Post by IncreasePosts »

sk11001 wrote: post #16
_fuchs wrote: post #2

The prompts: https://docs.anthropic.com/en/release-notes/system-prompts

It's interesting that they're in the 3rd person - "Claude is", "Claude responds", instead of "you are", "you respond".

Why not first person? I assumed the system prompt was like internal monologue.
Post reply on HN