11–20 of 27 posts
One strategy(not directly related to ChatGPT) is to use two models, one for extraction/generation and the other "challenger" to verify the extracted answer. Refer: https://docs.unstract.com/editions/cloud_edition#llmchalleng...
2. Explicitly call out null conditions (e.g. return { “results”: [] })
3. Use multiple prompts, one to “think”/explain and then one to transform the result
4. Don’t use function calling to get structured output, just use JSON mode
One non-obvious trick we use is to tell the LLM what it said previously as a system messages, not just as user messages, even if the LLM didn’t actually output that specific text.
Explicitly allow it the option to be unsure, e.g. "If you do not know the answer, respond with 'none'" or "If you are unsure of the answer, just say that", etc. Otherwise it does what humans do when asked interview questions, they bullshit because if you bullshit is a 20% chance of landing the job, whereas if you say "I don't know" there is a 0% chance of landing the job. The kind of RLHF training that was put into C…
Explicitly allow it the option to be unsure, e.g. "If you do not know the answer, respond with 'none'" or "If you are unsure of the answer, just say that", etc. Otherwise it does what humans do when asked interview questions, they bullshit because if you bullshit is a 20% chance of landing the job, whereas if you say "I don't know" there is a 0% chance of landing the job. The kind of RLHF training that was put into C…
This assumes the it “knows” what it knows ahead of time which isn’t true. It’s why models struggle to tell you how many Rs are in strawberry or why it can’t tell you how many words are in its response. Also doesn’t really have a concept of certainty beyond perhaps logprobs which are flimsy indicators at best and the model isn’t inherently aware of them
If you tell it that it can be unsure, the likelihood that an answer is correct increases. Make of that philosophically what you will. If it works it works.
My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quite recently, would answer yes, often with the most bizarre reasoning. The way to resolve it on most models over a certain size is a common tactic used with LLMs: ask the LLM to "think through your answer first". For example, you have a system prompt akin to: "B…
Llama3-8b-8192, Gemma2-9b-lt, Llama-3.1-70b-Versatile - All Said Yes
Llama-3.1-8b-Instant:
King Henry VIII had three children who survived to adulthood: Mary I (Mary Tudor, also known as Bloody Mary), Elizabeth I, and Edward VI.
Mary I had one child, a son named Edward VI, who became King of England at the age of 9 after Henry VIII's death. Edward VI died at the age of 15, so he did not have any children.
Elizabeth I never married and did not have any children.
Therefore, King Henry VIII did not have any grandchildren.
???!
Llama3-70b-8192: Also similar to above.
Of course if I ask again I may get a different answer!