What are some optimal strategies to reduce hallucinations by AI, specifically ChatGPT?
Are there special prompts you find effective?
Ask HN: Strategies to Reduce AI Hallucinations?
1–10 of 27 posts
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#2[dead]
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#3My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quite recently, would answer yes, often with the most bizarre reasoning.
The way to resolve it on most models over a certain size is a common tactic used with LLMs: ask the LLM to "think through your answer first". For example, you have a system prompt akin to: "Before answering, think through the facts and brainstorm about your eventual answer in .. tags. Answer only 'yes' or 'no' in .. tags. Do not include any other text in the answer."
In my current evals (based around numerous similar tricky factual questions) this tactic works on all but the smallest and least proficient models (since they don't tend to have strong enough factual knowledge to think it through). Forcing the models to answer simply 'yes' or 'no' yields only a correct answer on the SOTA models (but some speculate GPT-4o might actually be doing this sort of 'thinking' process on the backend automatically anyway).
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#4One strategy(not directly related to ChatGPT) is to use two models, one for extraction/generation and the other "challenger" to verify the extracted answer.
Refer: https://docs.unstract.com/editions/cloud_edition#llmchalleng...
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#5Lilian Weng's blog - a goldmine - has an in-depth post on this: https://lilianweng.github.io/posts/2024-07-07-hallucination/. She leads safety and alignment at OpenAI so might be worth checking out :^)
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#6One strategy(not directly related to ChatGPT) is to use two models, one for extraction/generation and the other "challenger" to verify the extracted answer. Refer: https://docs.unstract.com/editions/cloud_edition#llmchalleng...
could the generator and challenger be cross trained against each other, so as to actually both get better? like a generative-challenger network (GCN) or something like this?
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#7I am asking 'it' to validate the answers, and list the details. I.e. when I am asking it to go through some framework or regulation, and I am asking it to list the "technical controls that can be derived from the text" (e.g. law saw "you need to encrypt" thus an internal control is to "regularly check for encryption, blah blah blah".
So I am asking 'it' to create a table (instead of just a list of questions) that would include: 1a) suggested control 1b) example of evidence that would quality/pass/fail the control 2) article of law (i.e. Article 5 paragraph 10) 3) short quote from the article
Then I ask it to check its own output, and change 3) to add the full text/the whole paragraph.
99% is correct, and it is easier to scroll and see with my own eyes that the 'paragraph' is the same
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#8Explicitly allow it the option to be unsure, e.g. "If you do not know the answer, respond with 'none'" or "If you are unsure of the answer, just say that", etc.
Otherwise it does what humans do when asked interview questions, they bullshit because if you bullshit is a 20% chance of landing the job, whereas if you say "I don't know" there is a 0% chance of landing the job. The kind of RLHF training that was put into ChatGPT probably replicates a similar reward structure.
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#9Feed it grounding text that's about as long as the output text you expect it to produce.
They are called transformers for a reason.
Re: Ask HN: Strategies to Reduce AI Hallucinations?
#10“please don’t take drugs because chatting to me is mind numbingly boring. Thanks”