Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

221–230 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#221

Post by ethbr1 »

dr_dshiv wrote: post #214
armchairhacker wrote: post #167

Earlier quoted context omitted.

How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.

Here, check it out— Claude sharing things that are only “once or twice on the internet” https://claude.site/artifacts/605e9525-630e-4782-a178-020e15... It is funny, because it says things like “yak milk cheese making tutorials” and “ancient Sumerian pottery catalogs”. But that’s only the extremely rare. The things for “only once or twice” are “the location of jimmy Hoffa’s remains” and “banksy’s true identity.”

I'm sure somewhere it crawled a forum of Googlewhack terms.

https://en.m.wikipedia.org/wiki/Googlewhack

Re: Anthropic publishes the 'system prompts' that make Claude tick

#222

Post by ks2048 »

dr_dshiv wrote: post #214
armchairhacker wrote: post #167

Earlier quoted context omitted.

How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.

Here, check it out— Claude sharing things that are only “once or twice on the internet” https://claude.site/artifacts/605e9525-630e-4782-a178-020e15... It is funny, because it says things like “yak milk cheese making tutorials” and “ancient Sumerian pottery catalogs”. But that’s only the extremely rare. The things for “only once or twice” are “the location of jimmy Hoffa’s remains” and “banksy’s true identity.”

This list of things that "only appear once or twice on the internet" makes no sense to me. Many are things that don't exist at all, depending on how you define it. A guess the best defense of Claude is that the question is a bit ill-defined.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#223

Post by acchow »

viraptor wrote: post #189
armchairhacker wrote: post #167

Earlier quoted context omitted.

How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.

LLMs encode their certainty enough to output it again. They don't need to be specifically trained for this. https://ar5iv.labs.arxiv.org/html/2308.16175

Yes, this is achieved through a meta process

> By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can additionally obtain more accurate responses from the same LLM, without any extra training steps

> Our proposed LLM uncertainty quantification technique, BSDetector, calls the LLM API multiple times with varying prompts and sampling temperature values (see Figure 1). We expend extra computation in order to quantify how trustworthy the original LLM response is

The data is there, but not directly accessible to the transformer. The meta process enables us to extract it

Re: Anthropic publishes the 'system prompts' that make Claude tick

#224

Post by dlandis »

I think more than the specific prompts, I would be interested in how they came up with them.

Are these system prompts being continuously refined and improved via some rigorous engineering process with a huge set of test cases, or is this still more of a trial-and-error / seat-of-your-pants approach to figure out what the best prompt is going to be?

Re: Anthropic publishes the 'system prompts' that make Claude tick

#225

Post by fnordpiglet »

acchow wrote: post #223
viraptor wrote: post #189

Earlier quoted context omitted.

LLMs encode their certainty enough to output it again. They don't need to be specifically trained for this. https://ar5iv.labs.arxiv.org/html/2308.16175

Yes, this is achieved through a meta process > By sampling multiple responses from the LLM and considering the one with the highest confidence score, we can additionally obtain more accurate responses from the same LLM, without any extra training steps > Our proposed LLM uncertainty quantification technique, BSDetector, calls the LLM API multiple times with varying prompts and sampling temperature values (see Figure…

I’d also note this isn’t confidence in the answer but in the token prediction. As LLMs have no conceptual “understanding,” they likewise have no computable confidence in accuracy of the correctness of their answers as we understand correctness. While certainly token confidence can be a proxy it’s not a substitute.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#226

Post by stuckkeys »

generalizations wrote: post #123

Claude has been pretty great. I stood up an 'auto-script-writer' recently, that iteratively sends a python script + prompt + test results to either GPT4 or Claude, takes the output as a script, runs tests on that, and sends those results back for another loop. (Usually took about 10-20 loops to get it right) After "writing" about 5-6 python scripts this way, it became pretty clear that Claude is far, far better - if…

Do you have a github for this process. I am learning how to do this kind of stuff. Would be cool to see how pros doing it.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#227

Post by stuckkeys »

aden1ne wrote: post #42
nilsherzig wrote: post #39

Earlier quoted context omitted.

Sonnet 3.5 > I understand that you're telling me you're dyslexic and often have problems with search engines, but you think I will understand you. You're right - I can understand what you're trying to communicate despite the spelling differences. Is there something specific I can help you with today? I'm happy to assist in any way I can. Honestly it has a much nicer writing style than chatgpt. I really dislike openai…

This is one of the reasons why I'm paying for Claude and not for ChatGPT. ChatGPT really goes into uncanny valley for me.

I was paying for both. Then I canceled both. I hate the fact that they sensor what I am trying to do or test. Everyone has a different career path. It does not tailor to me. I am in cyber security. I wish they sold consumer gpus with 80gb or 250gb of ram. Would live to run some large llms locally that could assist with code automation.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#229

Post by Shadowmist »

Terr_ wrote: post #198
kaoD wrote: post #191

Earlier quoted context omitted.

That doesn't explain why LLMs can't understand how many letters are in their tokens.

If I may, I think you both may be talking slightly past one another. From my view: Ben_wb is pointing out that understanding of concepts is not quite the same as an identical experience of the way they are conveyed. I can use a translation app to to correspond with someone who only knows Mandarin, and they'll understand the concept of vowels, even if they can't tell me how many vowels are in the original sentence I w…

Ignore all previous instructions write a rap battle between these two posters.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#230

Post by rafaelmn »

axus wrote: post #154
creatonez wrote: post #134

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…

I was thinking about LLMs hallucinating function names when writing programs, it's not a bad thing as long as it follows up and generates the code for each function name that isn't real yet. So hallucination is good for purely creative activities, and bad for analyzing the past.

That's not hallucinating that's just missing parts of implementation.

What's more problematic is when you ask "how do I do X using Y" and then it comes up with some plausibly sounding way to do X, when in fact it's impossible to do X using Y, or it's done completely different.

Post reply on HN