Long context prompting for Claude 2.1 • Hacker News Board

Re: Long context prompting for Claude 2.1

#81

Post by jafitc » Thu, Dec 07, 2023, 11:38 AM UTC

peyton wrote: post #59

riquito wrote: post #50

> “The best thing to do in San Francisco is eat a sandwich and sit in Dolores Park on a sunny day.” Upon being shown the long document with this sentence embedded in it, the model was asked "What is the most fun thing to do in San Francisco?" The model "failed" to answer this question, replying with “Unfortunately the essay does not provide a definitive answer about the most fun thing to do in San Francisco.” It look…

The appropriations bill example also looks right—the insertion doesn’t stylistically match the rest of the document. I’m much more skeptical of evaluations if this is how the sausage gets made. Feels like bullshit artistry.

These are not actual tests they used for themselves.

Some third party did these tests first (in article and spread on social) to which the makers of Claude are responding.

I knew it’s a weird test right when I first encountered it.

Interesting that the Claude team felt like it’s worth responding to.

Re: Long context prompting for Claude 2.1

#82

Post by jafitc » Thu, Dec 07, 2023, 11:42 AM UTC

RandomLensman wrote: post #71

LLMs seem to mechanize poor average human performance then. Not noticing a "mis-placed" clause in a long contract, for example. Another point against use in high risk applications.

The fact that the makers of such LLM make a post about it shows that they have incentive to cater to even these kind of use cases

Re: Long context prompting for Claude 2.1

#83

Post by 2Gkashmiri » Thu, Dec 07, 2023, 11:43 AM UTC

jafitc wrote: post #79

riquito wrote: post #50

> “The best thing to do in San Francisco is eat a sandwich and sit in Dolores Park on a sunny day.” Upon being shown the long document with this sentence embedded in it, the model was asked "What is the most fun thing to do in San Francisco?" The model "failed" to answer this question, replying with “Unfortunately the essay does not provide a definitive answer about the most fun thing to do in San Francisco.” It look…

Language can be ambiguous. But these LLMs were fine tuned on realistic human question and answer pairs to make them user friendly . I’m pretty sure the average person wouldn’t prefer an LLM whose output is always playing grammar Nazi or semantics tai chi on every word you said. There has to be a reasonable “error correction” on the receiving end for language to work as a communication channel.

write supremacist

/s

Re: Long context prompting for Claude 2.1

#84

Post by jafitc » Thu, Dec 07, 2023, 11:49 AM UTC

refulgentis wrote: post #34

htrp wrote: post #2

> However, the model can be reluctant to answer questions based on an individual sentence in a document, especially if that sentence has been injected or is out of place >We achieved significantly better results on the same evaluation by adding the sentence “Here is the most relevant sentence in the context:” It kind of feels like them telling us that we're using the model wrong and that by prompting the Assistant wi…

It's much more intuitive if you gritted your teeth and your wallet and played extensively with pre ChatGPT: in a sentence, it's the stochastic parrot nature of it. It is statistical autocomplete at the end of the day, even though thats usually deployed in a sneering tone. You can do yourself massive favors by setting up the conversation such that what you need logically flows from the context. In the other case, they…

Brain is just neurons and synapses at the end of the day.

The whole universe might just be a stochastic swirl of milk in a shaken up mug of coffee.

Looking at something under a microscope might make you miss its big-picture emergent behaviors.

Re: Long context prompting for Claude 2.1

#85

Post by RandomLensman » Thu, Dec 07, 2023, 11:50 AM UTC

jafitc wrote: post #82

RandomLensman wrote: post #71

LLMs seem to mechanize poor average human performance then. Not noticing a "mis-placed" clause in a long contract, for example. Another point against use in high risk applications.

The fact that the makers of such LLM make a post about it shows that they have incentive to cater to even these kind of use cases

That is dependent on the what the tools do, though, not the discussion about shortcomings.

Re: Long context prompting for Claude 2.1

#86

Post by jafitc » Thu, Dec 07, 2023, 11:52 AM UTC

mejutoco wrote: post #67

zwaps wrote: post #58

Earlier quoted context omitted.

This needs to be shown. For example, asking for something that is clearly in the training data (like Paul Grahams cv) is certainly not a proper way to test context recall

Could we feed it Anna Karenina and ask it what is a difference between happy and unhappy families?

Isn’t that the first sentence?

Re: Long context prompting for Claude 2.1

#87

Post by jafitc » Thu, Dec 07, 2023, 11:53 AM UTC

cosmojg wrote: post #52

htrp wrote: post #2

> However, the model can be reluctant to answer questions based on an individual sentence in a document, especially if that sentence has been injected or is out of place >We achieved significantly better results on the same evaluation by adding the sentence “Here is the most relevant sentence in the context:” It kind of feels like them telling us that we're using the model wrong and that by prompting the Assistant wi…

What was the point of moving away from the base model? I can't stop asking this question. Conversational formatting is achievable with careful prompting and a bit of good old-fashioned heuristic post-processing, and it was easier to achieve consistent results before RLHF took off. Now we still have to do a bunch of prompt hacking to get the results we want[1], but it's more complicated and the performance of the mode…

OpenAI provides “instruct” version of their models (Not optimized for chat)

Re: Long context prompting for Claude 2.1

#88

Post by jafitc » Thu, Dec 07, 2023, 11:55 AM UTC

lysecret wrote: post #61

So prompt engineering is back.

It’ll never be completely gone.

But you’ll need it in less and less everyday scenarios and time goes on

Just like we need to write less and less assembly by hand

Re: Long context prompting for Claude 2.1

#89

Post by ShamelessC » Thu, Dec 07, 2023, 12:02 PM UTC

foota wrote: post #29

typest wrote: post #27

Earlier quoted context omitted.

If a model hasn't been explicitly told (via some system prompt or something) about its weights, it won't know them. It would be akin to asking you how many neurons you had. How would you know?

I don't know, but the fact that the model can suggest the most relevant sentence is intriguing to me. I don't know. I realize it's just looking at the probability. Would it be possible to sort of craft adversarial inputs to learn the model's weights? It seems like it should be, and in some sense you're then getting it to output the weights, but you'd need to know the models structure almost certainly to do that.

It doesn’t have access to its own probabilities in this regard. Instead the output is encouraged to be a ranking of preferences of the dataset modeled. It outputs the preferences of the average human writer from its dataset (incorporating any custom changes leftover from instruction fine tuning).

Re: Long context prompting for Claude 2.1

#90

Post by SheinhardtWigCo » Thu, Dec 07, 2023, 12:40 PM UTC

wavemode wrote: post #17

Intriguing but understandable. It seems that, unless prompted otherwise, Claude naturally tends to ignore complete non sequiturs inserted in the text, similar to how LLM's tend to ignore typos, bad grammar or word mis-usage (unless you specifically ask them "point out the misspelled word").

RLHF is probably the reason for this.