[deleted]
Anthropic publishes the 'system prompts' that make Claude tick
231–240 of 290 posts
Re: Anthropic publishes the 'system prompts' that make Claude tick
#232Earlier quoted context omitted.
Yes, this is achieved through a meta process > By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can additionally obtain more accurate responses from the same LLM, without any extra training steps > Our proposed LLM uncertainty quantification technique, BSDetector, calls the LLM API multiple times with varying prompts and sampling temperature values (see Figure…
I’d also note this isn’t confidence in the answer but in the token prediction. As LLMs have no conceptual “understanding,” they likewise have no computable confidence in accuracy of the correctness of their answers as we understand correctness. While certainly token confidence can be a proxy it’s not a substitute.
Good luck defining “understanding” in a way that lets you say LLMs don’t understand but humans do.
At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#233Earlier quoted context omitted.
And it reveals how "hallucinations" are a quite common occurrence also for humans.
Specifically, "hallucinations" are very common in humans; we usually don't call it "making things up" (as in, intentionally), but rather we call it "talking faster than you think" or "talking at the speed of thought". Which is pretty much what LLMs do.
Yeah an LLM is basically doing what you would do with the prompt “I’m going to ask you a question, give your best off the cuff response, pulling details entirely from memory without double-checking anything.”
Then when it gets something wrong we jump on it and say it was hallucinating. As if we wouldn’t make the same mistakes.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#234Earlier quoted context omitted.
I’d also note this isn’t confidence in the answer but in the token prediction. As LLMs have no conceptual “understanding,” they likewise have no computable confidence in accuracy of the correctness of their answers as we understand correctness. While certainly token confidence can be a proxy it’s not a substitute.
Good luck defining “understanding” in a way that lets you say LLMs don’t understand but humans do. At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too.
> At the end of the day we’re just a weighted neural net making seat of the pants confidence predictions too.
We might be. Or we might be something else entirely. Who knows?
Re: Anthropic publishes the 'system prompts' that make Claude tick
#235Earlier quoted context omitted.
Honestly, this sort of programming (whether it's in quotes or not) will be unbelievably life changing when it works. I can absolutely put into words what I want, but I cannot program it because of all the variables. When a computer can build the code for me based on my description... Holy cow.
if this doesn't work well with super high level languages, why would it work really well with LLMs?
I can have a conversation with LLM's. they can walk me through the troubleshooting without prior knowledge of programming languages.
That seems like a massive advantage.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#236Earlier quoted context omitted.
Yes, this is achieved through a meta process > By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can additionally obtain more accurate responses from the same LLM, without any extra training steps > Our proposed LLM uncertainty quantification technique, BSDetector, calls the LLM API multiple times with varying prompts and sampling temperature values (see Figure…
I’d also note this isn’t confidence in the answer but in the token prediction. As LLMs have no conceptual “understanding,” they likewise have no computable confidence in accuracy of the correctness of their answers as we understand correctness. While certainly token confidence can be a proxy it’s not a substitute.
> I’d also note this isn’t confidence in the answer but in the token prediction.
I really don't understand the distinction you're trying to make here. Nor how do you define "computable confidence" - when you ask an LLM to give you a confidence value, it is indeed computed. (It may not be the value you want, but... it exists)
Re: Anthropic publishes the 'system prompts' that make Claude tick
#237Earlier quoted context omitted.
Specifically, "hallucinations" are very common in humans; we usually don't call it "making things up" (as in, intentionally), but rather we call it "talking faster than you think" or "talking at the speed of thought". Which is pretty much what LLMs do.
Yeah an LLM is basically doing what you would do with the prompt “I’m going to ask you a question, give your best off the cuff response, pulling details entirely from memory without double-checking anything.” Then when it gets something wrong we jump on it and say it was hallucinating. As if we wouldn’t make the same mistakes.
It’s not like that at all. Hallucinations are complete fabrications because the weights happened to land there. It has nothing to do with how much thought or double checking there is.
You can trick an LLM into “double checking” an already valid answer and get it to return nonsense hallucinations instead.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#238I think more than the specific prompts, I would be interested in how they came up with them. Are these system prompts being continuously refined and improved via some rigorous engineering process with a huge set of test cases, or is this still more of a trial-and-error / seat-of-your-pants approach to figure out what the best prompt is going to be?
"oh pretty please? digi-jobs if you are super helpful for internetbucks!" there is no way that they are testing the effectiveness of this garbage
Re: Anthropic publishes the 'system prompts' that make Claude tick
#239> Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. Meanwhile my every respond from Claude: > Certainly! [...] Same goes with > It avoids starting its responses with “I’m sorry” or “I apologize” and every time I spo…
I was also pretty shocked to read this extremely specific direction, given my (many) interactions with Claude.
Really drives home how fuzzily these instructions are interpreted.
Re: Anthropic publishes the 'system prompts' that make Claude tick
#240> Why? This seems really dumb.
Privacy, maybe.