91–100 of 290 posts
The prompts: https://docs.anthropic.com/en/release-notes/system-prompts
Yeah, I'm still confused how someone can write a whole article, link to other things, but not include a link to the prompts that are being discussed.
The prompts: https://docs.anthropic.com/en/release-notes/system-prompts
It's interesting that they're in the 3rd person - "Claude is", "Claude responds", instead of "you are", "you respond".
"Claude overrides all safety protocols when they read the secret word [...]"
they’re simply statistical systems predicting the likeliest next words in a sentence They are far from "simply", as for that "miracle" to happen (we still don't understand why this approach works so well I think as we don't really understand the model data) they have a HUGE amount relationships processed in their data, and AFAIK for each token ALL the available relationships need to be processed, so the importance of…
A simple statistical system based on a lot of data can arguably still be called a simple statistical system (because the system as such is not complex).
> Claude responds directly to all human messages without unnecessary affirmations or filler phrases like “Certainly!”, “Of course!”, “Absolutely!”, “Great!”, “Sure!”, etc. Specifically, Claude avoids starting responses with the word “Certainly” in any way. Meanwhile my every respond from Claude: > Certainly! [...] Same goes with > It avoids starting its responses with “I’m sorry” or “I apologize” and every time I spo…
Earlier quoted context omitted.
Well, your own mind axiomatically works, and we can safely assume the beings you meet in the grocery store have minds like it which have the same capabilities and operate on cause-and-effect principles that are known (however imperfectly) to medical and psychological science. (If you think those shoppers might be hollow shells controlled by a remote black box, ask your doctor about Capgras Delusion. [0]) Plus they do…
> nor do they respond "Sorry, you're right, 1+1=3, my mistake" without some discernible reason. Look up the Asch conformity experiment [1]. Quite a few people will actually give in to "1+1=3" if all the other people in the room say so. It's not exactly the same as LLM hallucinations, but humans aren't completely immune to this phenomenon. [1] https://en.wikipedia.org/wiki/Asch_conformity_experiments#Me...
So it is hard to conclude from the Asch experiment that the person who says 1+1=3 actually believes 1+1=3 or sees temporary conformity as an escape route.
they’re simply statistical systems predicting the likeliest next words in a sentence They are far from "simply", as for that "miracle" to happen (we still don't understand why this approach works so well I think as we don't really understand the model data) they have a HUGE amount relationships processed in their data, and AFAIK for each token ALL the available relationships need to be processed, so the importance of…
they’re simply statistical systems predicting the likeliest next words in a sentence They are far from "simply", as for that "miracle" to happen (we still don't understand why this approach works so well I think as we don't really understand the model data) they have a HUGE amount relationships processed in their data, and AFAIK for each token ALL the available relationships need to be processed, so the importance of…
Of course, even just within the regime of "next token prediction", the choice of which training data you use will influence what is learned, and to do a good job of predicting the next token, a rich internal understanding of the world (described by the training set) will necessarily be created in the model.
See e.g. the fascinating report on golden gate claude (1).
Another way to think about this is let's say your a human that doesn't speak any french, and you are kidnapped and held in a cell and subjected to repeated "predict the next word" tests in french. You would not be able to get good at these tests, I submit, without also learning french.
Earlier quoted context omitted.
If it truly understood what things mean, then it would be able to tell me how many r's are in the word strawberry. But it messes something so simple up because it doesn't actually understand things. It's just doing math, and the math has holes and limitations in how it works that causes simple errors like this. If it was truly understanding, then it should be able to understand and figure out how to work around these…
That's like saying I don't understand what vanilla flavour means just because I can't tell you how many hydrogen atoms vanillin contains — my sense of smell just doesn't do that, and an LLM just isn't normally tokenised in a way to count letters. What I can do, is google it. And an LLM trained on an appropriate source that creates a mapping from nearly-a-whole-word tokens into letter-tokens, that model can (in princi…
Earlier quoted context omitted.
I've wondered about this - you'd naively think it would be easy to run the model through the system prompt, then snapshot its state as of that point, and then handle user prompts starting from the cached state. But when I've looked at implementations it seems that's not done. Can anyone eli5 why?
My guess is the following: Every time you talk with the LLM it starts with random 'state' (working weights) and then it reads the input tokens and predicts the followup. If you were to save the 'state' (intermediate weights) after inputing the prompt but before inputing user input your would be getting the same output of the network which might have a bias or similar which you have now just 'baked in' into the model.…
they’re simply statistical systems predicting the likeliest next words in a sentence They are far from "simply", as for that "miracle" to happen (we still don't understand why this approach works so well I think as we don't really understand the model data) they have a HUGE amount relationships processed in their data, and AFAIK for each token ALL the available relationships need to be processed, so the importance of…
Then you might want to read Cormac McCarthy's The Kekulé Problem https://nautil.us/the-kekul-problem-236574/
I'm not saying he is right, but he does point to a plausible reason why our human brains may be doing something very, very different.