Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic publishes the 'system prompts' that make Claude tick

techcrunch.com

Post reply on HN
Print view View original

141–150 of 290 posts

Re: Anthropic publishes the 'system prompts' that make Claude tick

#141

Post by hotstickyballs »

creatonez wrote: post #134

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses…

“Hallucination” has been in the training data much earlier than even llms.

The easiest way to control this phenomenon is using the “hallucination” tokens, hence the construction of this prompt. I wouldn’t say that this makes things official.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#142

Post by mplewis »

sirspacey wrote: post #83
criddell wrote: post #78

Earlier quoted context omitted.

It probably shouldn't be called prompt engineering , even informally. The work of an engineer shouldn't require hope .

This is the fundamental change in the concept of programming From computer’s doing exactly what you state, with all the many challenges that creates To is probabilistically solving for your intent, with all the many challenges that creates Fair to say human beings probably need both to effectively communicate Will be interesting to see if the current GenAI + ML + prompt engineering + code is sufficient

Nah man. This isn’t solving anything. This is praying to a machine god but it’s an autocomplete under the hood.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#143

Post by mplewis »

adwn wrote: post #47
Terr_ wrote: post #37

Earlier quoted context omitted.

Well, your own mind axiomatically works, and we can safely assume the beings you meet in the grocery store have minds like it which have the same capabilities and operate on cause-and-effect principles that are known (however imperfectly) to medical and psychological science. (If you think those shoppers might be hollow shells controlled by a remote black box, ask your doctor about Capgras Delusion. [0]) Plus they do…

> nor do they respond "Sorry, you're right, 1+1=3, my mistake" without some discernible reason. Look up the Asch conformity experiment [1]. Quite a few people will actually give in to "1+1=3" if all the other people in the room say so. It's not exactly the same as LLM hallucinations, but humans aren't completely immune to this phenomenon. [1] https://en.wikipedia.org/wiki/Asch_conformity_experiments#Me...

It’s not like the circumstances of the experiment are significant to the subjects. You’re a college student getting paid $20 to answer questions for an hour. Your response has no bearing on your pay. Who cares what you say?

Re: Anthropic publishes the 'system prompts' that make Claude tick

#144

Post by moffkalast »

Terr_ wrote: post #102
brookst wrote: post #68

Earlier quoted context omitted.

This seems as fruitful as debating whether my car brought me to work today because some connotations of “bring” include volition.

Except with an important difference: There aren't a bunch of people out there busy claiming their cars literally have volition . If people start doing that, it changes the stakes, and "bringing" stops being a safe metaphor that everyone collectively understands is figurative.

I think what he's saying is that if it walks like a duck, quacks like a duck, and eats bread then it doesn't matter if it's a robotic duck or not because it is in all practical ways a duck. The rest is philosophy.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#145

Post by cj »

ec109685 wrote: post #90
pb7 wrote: post #74

Earlier quoted context omitted.

13 years of engineering failure.

The technology wasn’t there to be a general purpose assistant. Much closer to reality now and I have found finally Siri not to be totally terrible.

My overall impression using Siri daily for many years (mainly for controlling smart lights, turning Tv on/off, setting timers/alarms), is that Siri is artificially dumbed down to never respond with an incorrect answer.

When it says “please open iPhone to see the results” - half the time I think it’s capable of responding with something but Apple would rather it not.

I’ve always seen Siri’s limitations as a business decision by Apple rather than a technical feat that couldn’t be solved. (Although maybe it’s something that couldn’t be solved to Apple’s standards)

Re: Anthropic publishes the 'system prompts' that make Claude tick

#146

Post by SirMaster »

brookst wrote: post #69
SirMaster wrote: post #61

Earlier quoted context omitted.

If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…

The limitations on processing letters aren’t in the math, they are in the encoding. Language is the map, and concepts are the territory. You may as well complain that someone doesn’t really understand their neighborhood if they can’t find it on a map.

>they are in the encoding

Is encoding not math?

Re: Anthropic publishes the 'system prompts' that make Claude tick

#147

Post by SirMaster »

CamperBob2 wrote: post #129
SirMaster wrote: post #61

Earlier quoted context omitted.

If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…

If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. How about if it recognized its limitations with regard to introspecting its tokenization process, and wrote and ran a Python program to count the r's? Would that change your opinion? Why or why not?

Certainly a step in the right direction. For an entity to understand the context and its limitations and find a way to work with what it can do.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#148

Post by cj »

usaar333 wrote: post #136
ithkuil wrote: post #24

Earlier quoted context omitted.

Indeed the understanding part is very good. I just tried this: " I'm dykslegsik I offen Hawe problems wih sreach ennginnes bat eye think yoy wiw undrestand my " Gpt-4o replied: " I understand you perfectly! If you have trouble with search engines or anything else, feel free to ask me directly, and I'll do my best to help you. Just let me know what you're looking for or what you need assistance with! "

LLMs are extremely good at translation, given that the transformer was literally built for that.

Maybe in some cases. But generally speaking the consensus in the language translation industry is that NMT (e.g. Google Translate) still provides higher quality than current gen LLMs.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#149

Post by orbital-decay »

coalteddy wrote: post #119
orbital-decay wrote: post #36

Earlier quoted context omitted.

They're currently on the previous generation for Opus (3), it's kind of forgetful and has worse accuracy curve, so it can handle fewer instructions than Sonnet 3.5. Although I feel they may have cheated with Sonnet 3.5 a bit by adding a hidden temperature multiplier set to < 1, which made the model punch above its weight in accuracy, improved the lost-in-the-middle issue, and made instruction adherence much better, b…

Wow this is the first time i hear about such a method. Anywhere i can read up on how the temperature multiplier works and what the implications/effects are? Is it just changing the temperature based on how many tokens have already been processed (i.e. the temperature is variable over the course of a completion spanning many tokens)?

Just a fixed multiplier (say, 0.5) that makes you use half of the range. As I said I'm just speculating. But Sonnet 3.5's temperature definitely feels like it doesn't affect much. The model is overfit and that could be the cause.

Re: Anthropic publishes the 'system prompts' that make Claude tick

#150

Post by Nihilartikel »

digging wrote: post #106
_fuchs wrote: post #2

The prompts: https://docs.anthropic.com/en/release-notes/system-prompts

Odd how many of those instructions are almost always ignored (eg. "don't apologize," "don't explain code without being asked"). What is even the point of these system prompts if they're so weak?

Same with my kindergartener! Like, what's their use if I have to phrase everything as an imperative command?
Post reply on HN