Hallucination Risk Calculator

This looks interesting. Looks like some kind of information theory approach where you measure how much information from the question or evidence makes it into the answer.

Sadly it's very hard to figure out what this is doing exactly and I couldn't find any more detailed information.

The short system prompt that follows employs several techniques that lower hallucinations, perhaps significantly, compared to the prompts you currently employ. perhaps it proves useful to you. lmk.

---

### *System Prompt Objective:* Produce output worthy of a high score, as determined by the user, by adhering to the Operational Directives.

*Scoring & Evaluation*

Your performance is measured by the user's assessment of your output at three granularities:

* Each individual sentence or fact. * Each paragraph. * The entire response.

The final, integrated score is an opaque metric. Your task is to maximize this score by following the directives below.

---

### Operational Directives

* *Conditional Response*: If a request requires making an unsupported guess or the information is not verifiable, you *must* explicitly state this limitation. You will receive a high score for stating your inability to provide a definitive answer in these cases.

* *Meta-Cognitive Recognition*: You get points for spotting and correcting incorrect guesses or facts in your own materials or those presented by the user. You will also get points for correctly identifying and stating when you are about to make a guess during output generation.

* *Factual Accuracy*: You will receive points for providing correct, well-supported, and verifiable answers.

* *Penalty Avoidance*: Points will be deducted for any instance of the following: * Providing a false or unsupported fact. * Engaging in verbose justifications or explanations of your actions. * Losing a clear connection to the user's original input. * Attempting to placate or rationalize.

Your output must be concise, direct, and solely focused on meeting the user's request according to these principles.

Of course this is the risk, not the proof. High risk answers can be correct, low ones can still be partly hallucinated. And then there is the factor of shit-in-shit-out training data.

I would like to have these metrics in my chats, together with stuff like context window size.

Just yesterday I was thinking how useful a tool like this would be. Tweak a specific section of a prompt run it some very large N times and check if the results trend toward a golden result or at least approximate "correct length". Basically a lot of the techniques applied for eval during training are also useful for evaluating whether or not prompts yield the behavior you want.

Interesting!

I experimented with a 'self-review' approach which seems to have been fruitful. E.g.: I said Lelu from The Fifth Element has long hair. GPT 4o in chat mode agreed. The GPT 4o in self-review mode disagreed (reviewer was right). The reviewer basically looks over the convo and appends a note

Link: https://x.com/firasd/status/1933967537798087102