Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#1

Post by eskibars » Mon, Nov 06, 2023, 7:11 PM UTC

Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems.

What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and some specific examples of these summarization hallucinations), and a GitHub repository containing our evaluation from the most popular generative LLMs available today. Links to all of them are referenced in the blog here, but for the technical audience here, the most interesting additional links might be:

- https://huggingface.co/vectara/hallucination_evaluation_mode...

- https://github.com/vectara/hallucination-leaderboard

We hope that releasing these under a truly open source license and detailing the methodology, we hope to increase the viability of anyone really quantitatively measuring and improving the generative LLMs they're publishing.

Show HN: Open-source model and scorecard for measuring hallucinations in LLMs
vectara.com

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#2

Post by simonhughes22 » Mon, Nov 06, 2023, 7:33 PM UTC

I worked on the model with our research team. Recently featured in this NYT (https://www.nytimes.com/2023/11/06/technology/chatbots-hallu.... Post here to AMA. We are also looking for collaborators to help us maintain this model and make it the best it can be. Let us know if you want to help

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#3

Post by vinni2 » Mon, Nov 06, 2023, 8:38 PM UTC

Great work! Interesting to see Llama 2 7B is better than Llama 2 13B.

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#4

Post by Boerworz » Mon, Nov 06, 2023, 8:44 PM UTC

simonhughes22 wrote: post #2

I worked on the model with our research team. Recently featured in this NYT ( https://www.nytimes.com/2023/11/06/technology/chatbots-hallu... . Post here to AMA. We are also looking for collaborators to help us maintain this model and make it the best it can be. Let us know if you want to help

Hey, looks like your (very interesting) link got formatted incorrectly! Should be https://www.nytimes.com/2023/11/06/technology/chatbots-hallu..., right? :)

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#5

Post by awadallah » Mon, Nov 06, 2023, 8:46 PM UTC

I am CEO and one of cofounders of Vectara. We are very proud of the release of this open source eval model. We certainly would like to add more LLMs to the scorecard, and would love to collaborate with others to make the evaluation model even more accurate. Please reach out to bader@ or simon@ if interested.

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#6

Post by gcr » Mon, Nov 06, 2023, 9:11 PM UTC

simonhughes22 wrote: post #2

I worked on the model with our research team. Recently featured in this NYT ( https://www.nytimes.com/2023/11/06/technology/chatbots-hallu... . Post here to AMA. We are also looking for collaborators to help us maintain this model and make it the best it can be. Let us know if you want to help

interesting work, thanks! Not enough people studying this.

Do you have a whitepaper describing how you trained this hallucination detection model?

Is each row of the leaderboard the mean of the Vectara model's judgment of the 831 (article,summary) pairs, or was there any human rating involved? With so few pairs, it seems feasible that human ratings should be able to quantify how much hallucination is actually occurring.

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#7

Post by forrestbao » Mon, Nov 06, 2023, 9:33 PM UTC

vinni2 wrote: post #3

Great work! Interesting to see Llama 2 7B is better than Llama 2 13B.

A smaller model has less capacity and thus is less prone to overfitting, which is a reason of hallucination. Overfitting means that a model cannot adjust its output based on the input that is unseen during training.

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#8

Post by zer00eyz » Mon, Nov 06, 2023, 9:33 PM UTC

This link gives a lot more context for the less informed:

https://vectara.com/cut-the-bull-detecting-hallucinations-in...

I really want to know more about the hallucinations produced! Were some sources more likely to produce errors! I would be curious if other document sets were explored for use with this test and if different source material would change the results in a meaningful way.

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#9

Post by simonhughes22 » Mon, Nov 06, 2023, 9:36 PM UTC

Boerworz wrote: post #4

simonhughes22 wrote: post #2

I worked on the model with our research team. Recently featured in this NYT ( https://www.nytimes.com/2023/11/06/technology/chatbots-hallu... . Post here to AMA. We are also looking for collaborators to help us maintain this model and make it the best it can be. Let us know if you want to help

Hey, looks like your (very interesting) link got formatted incorrectly! Should be https://www.nytimes.com/2023/11/06/technology/chatbots-hallu... , right? :)

Yes thanks for fixing that.

Re: Show HN: Open-source model and scorecard for measuring hallucinations in LLMs

#10

Post by simonhughes22 » Mon, Nov 06, 2023, 9:38 PM UTC

forrestbao wrote: post #7

vinni2 wrote: post #3

Great work! Interesting to see Llama 2 7B is better than Llama 2 13B.

A smaller model has less capacity and thus is less prone to overfitting, which is a reason of hallucination. Overfitting means that a model cannot adjust its output based on the input that is unseen during training.

Yes. Just because the model is smaller doesn't always mean by default it's worse, as they may be trained for less time or on less data, which in some cases could be beneficial. The differences are small so may not be statistically significant. Plus the model is doing the evaluation, so while it's highly correlated with humans, a small difference like this may not mean that the the 7B model is necessarily better.