Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Long context prompting for Claude 2.1

anthropic.com

Post reply on HN
Print view View original

21–30 of 107 posts

Re: Long context prompting for Claude 2.1

#22

Post by s1artibartfast »

crawfordcomeaux wrote: post #9
jafitc wrote: post #5

Earlier quoted context omitted.

It’s not where , it’s how .

It's like they're saying "When we prompt the model asking for it to search in the way we want it to, it searches in the way we want it to. "

Same is true for most people.

Re: Long context prompting for Claude 2.1

#25

Post by jafitc »

mherdeg wrote: post #21

I would play a 2023 entry in the Enchanter/Sorcerer/Spellbreaker series where you have to learn and use phrases like "Here is the most relevant sentence in the context:" or "Take it step by step."

On a constructive note, these things will trickle down into the models. Bing for example already does "thinking" step that is hidden from the user.

Also see this quote from Ethan Mollick on twitter:

> I have a strong suspicion that “prompt engineering” is not going to be a big deal in the long-term & prompt engineer is not the job of the future

> AI gets easier. You can already see in Midjourney how basic prompts went from complex in v3 to easy in v4. Same with ChatGPT to Bing.

https://twitter.com/emollick/status/1627804798224580608?lang...

Re: Long context prompting for Claude 2.1

#26

Post by cl42 »

Wouldn't inserting a statement like "Here is the most relevant sentence in the context" predispose Claude to answer the question also increase the likelihood of hallucinations?

Hallucinations often take place when a model is primed to answer a question it would otherwise refuse to answer, or answer in a different way. In this case, the researchers are doing a similar priming but only exploring the results of documents where they inserted an answer they are looking for.

Re: Long context prompting for Claude 2.1

#27

Post by typest »

foota wrote: post #15
_pdp_ wrote: post #10

I wonder if you can preempt it but as part of the user message. For example: Human: {context} What is the most fun thing to do in San Francisco based on the context? Don't give in formation outside the document. Start with "Here is the most relevant sentence in the context:" Assistant: It just feels more natural to do it like that especially when constructing the prompt based on various factors.

I realize it's all just embeddings and probability blah blah blah... But this kind of meta prompting is really interesting to me. Can you ask a model about its weights?

If a model hasn't been explicitly told (via some system prompt or something) about its weights, it won't know them. It would be akin to asking you how many neurons you had. How would you know?

Re: Long context prompting for Claude 2.1

#29

Post by foota »

typest wrote: post #27
foota wrote: post #15

Earlier quoted context omitted.

I realize it's all just embeddings and probability blah blah blah... But this kind of meta prompting is really interesting to me. Can you ask a model about its weights?

If a model hasn't been explicitly told (via some system prompt or something) about its weights, it won't know them. It would be akin to asking you how many neurons you had. How would you know?

I don't know, but the fact that the model can suggest the most relevant sentence is intriguing to me. I don't know. I realize it's just looking at the probability. Would it be possible to sort of craft adversarial inputs to learn the model's weights? It seems like it should be, and in some sense you're then getting it to output the weights, but you'd need to know the models structure almost certainly to do that.
Post reply on HN