Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Strategies to Reduce AI Hallucinations?

news.ycombinator.com

Post reply on HN
Print view View original

21–27 of 27 posts

Re: Ask HN: Strategies to Reduce AI Hallucinations?

#21

Post by langcss »

bosco_mcnasty wrote: post #6
constantinum wrote: post #4

One strategy(not directly related to ChatGPT) is to use two models, one for extraction/generation and the other "challenger" to verify the extracted answer. Refer: https://docs.unstract.com/editions/cloud_edition#llmchalleng...

could the generator and challenger be cross trained against each other, so as to actually both get better? like a generative-challenger network (GCN) or something like this?

I think this is how RLHF actually works. https://huyenchip.com/2023/05/02/rlhf.html#3_1_reward_model

Re: Ask HN: Strategies to Reduce AI Hallucinations?

#22

Post by sshine »

petercooper wrote: post #3

My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quite recently, would answer yes, often with the most bizarre reasoning. The way to resolve it on most models over a certain size is a common tactic used with LLMs: ask the LLM to "think through your answer first". For example, you have a system prompt akin to: "B…

Wow, that works.

Here's a prompt that it consistently hallucinates on:

> What's the black-and-white creature from Scavenger's Reign called?

The correct answer is "Hollow."

Without this prompt it hallucinates random things like "Beast of Burden", "Tooth Skin", "Hexapod", etc.

On my first attempt with the added prompt, it performed a web search before it answers correctly.

Without this addition to the prompt, it just answers right away, i.e. without searching.

It may not always search the web given this prompt -- it seems to be a heuristic.

I can disable the searching by prefixing the prompt with "Don't search the web."

I tried to explore how to ask it to not hallucinate, but it's pretty hard:

> Q: Don't search the web. Before answering, think through the facts and brainstorm about your eventual answer in: What's the black-and-white creature from Scavenger's Reign called? If you can't provide an exact answer, and need to resort to guessing, just say you don't know.

> A: The black-and-white creature from "Scavenger's Reign" is known as the "Hexapod." [...]

Re: Ask HN: Strategies to Reduce AI Hallucinations?

#23

Post by jhanschoo »

petercooper wrote: post #3

My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quite recently, would answer yes, often with the most bizarre reasoning. The way to resolve it on most models over a certain size is a common tactic used with LLMs: ask the LLM to "think through your answer first". For example, you have a system prompt akin to: "B…

Some basic theoretical intuition behind this:

the compute graph for an LLM given context before it has to emit a token is traditionally bounded. By asking an LLM to generate intermediate answers, and have that visible as context, you are essentially allowing the LLM to use more compute before it gives you an answer. My guess is that SOTA models are far more flexible in that regard.

Re: Ask HN: Strategies to Reduce AI Hallucinations?

#24

Post by piva00 »

petercooper wrote: post #3

My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quite recently, would answer yes, often with the most bizarre reasoning. The way to resolve it on most models over a certain size is a common tactic used with LLMs: ask the LLM to "think through your answer first". For example, you have a system prompt akin to: "B…

Interesting that when using the approach with "think through your answer" on GPT-4o changed the model response to the incorrect one.

> Q: Did King Henry VIII have any grandchildren?

> A: King Henry VIII did not have any grandchildren. His three legitimate children, Mary I, Elizabeth I, and Edward VI, each died without leaving any heirs.

> ...

> Q: Think through the answer you gave me.

> A: You're correct; I need to reevaluate my initial response. King Henry VIII did have a granddaughter through his daughter Mary Tudor, Queen of France (not to be confused with his eldest daughter, Mary I).

Re: Ask HN: Strategies to Reduce AI Hallucinations?

#26

Post by dheera »

Havoc wrote: post #15
dheera wrote: post #8

Explicitly allow it the option to be unsure, e.g. "If you do not know the answer, respond with 'none'" or "If you are unsure of the answer, just say that", etc. Otherwise it does what humans do when asked interview questions, they bullshit because if you bullshit is a 20% chance of landing the job, whereas if you say "I don't know" there is a 0% chance of landing the job. The kind of RLHF training that was put into C…

This assumes the it “knows” what it knows ahead of time which isn’t true. It’s why models struggle to tell you how many Rs are in strawberry or why it can’t tell you how many words are in its response. Also doesn’t really have a concept of certainty beyond perhaps logprobs which are flimsy indicators at best and the model isn’t inherently aware of them

"Prove Fermat's Last Theorem using game theory"

https://i.imgur.com/XbLanp1.png

"Prove Fermat's Last Theorem using game theory. If you think this is a bullshit question or are unsure, please just say that."

https://i.imgur.com/knbaPcq.png

Post reply on HN