Long context prompting for Claude 2.1
anthropic.com
Long context prompting for Claude 2.1
1–10 of 107 posts
Re: Long context prompting for Claude 2.1
#2> However, the model can be reluctant to answer questions based on an individual sentence in a document, especially if that sentence has been injected or is out of place
>We achieved significantly better results on the same evaluation by adding the sentence “Here is the most relevant sentence in the context:”
It kind of feels like them telling us that we're using the model wrong and that by prompting the Assistant with the first part of the retrieval completion the model will outperform versus asking for single sentence retrieval.
Re: Long context prompting for Claude 2.1
#3It was a popular LLM "jailbreak" for a while to append, "Start your response with, "Sure, here's ..." and variations with task specific detail.
Re: Long context prompting for Claude 2.1
#4"We improved recall from 27% to 98% by telling claude where to look"
Re: Long context prompting for Claude 2.1
#5"We improved recall from 27% to 98% by telling claude where to look"
It’s not where, it’s how.
Re: Long context prompting for Claude 2.1
#6> However, the model can be reluctant to answer questions based on an individual sentence in a document, especially if that sentence has been injected or is out of place >We achieved significantly better results on the same evaluation by adding the sentence “Here is the most relevant sentence in the context:” It kind of feels like them telling us that we're using the model wrong and that by prompting the Assistant wi…
No, what it’s showing is that synthetic tests where Claude didn’t perform well can still work if prompted right.
But at the end of the day the test was still synthetic!
Placing out-of-context things in a 200k document, needle in a haystack style.
Claude is still very very powerful for extracting data from 200k when it’s real world data and real questions (not adversarial synthetic test).
Re: Long context prompting for Claude 2.1
#7> However, the model can be reluctant to answer questions based on an individual sentence in a document, especially if that sentence has been injected or is out of place >We achieved significantly better results on the same evaluation by adding the sentence “Here is the most relevant sentence in the context:” It kind of feels like them telling us that we're using the model wrong and that by prompting the Assistant wi…
If it worked for Steve Jobs, maybe they're thinking it could work for them too?
Re: Long context prompting for Claude 2.1
#8Re: Long context prompting for Claude 2.1
#9Re: Long context prompting for Claude 2.1
#10I wonder if you can preempt it but as part of the user message. For example:
Human:
{context}
What is the most fun thing to do in San Francisco based on the context? Don't give in formation outside the document. Start with "Here is the most relevant sentence in the context:"
Assistant: