Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why language models hallucinate

openai.com

Post reply on HN
Print view View original

131–140 of 242 posts

Re: Why language models hallucinate

#131

Post by didibus »

When tuning predictive models you always have to balance precision and recall because 100% accuracy is never going to happen.

In LLMs that balance shows up as how often the model hallucinates versus how often it says it doesn’t know. If you push toward precision you end up with a model that constantly refuses: What’s the X of Y? I don’t know. Can you implement a function that does K? I don’t know how. What could be the cause of G? I can’t say. As a user that gets old fast, you just want it to try, take a guess, let you be the judge of it.

Benchmarks and leaderboards usually lean toward recall because a model that always gives it a shot creates a better illusion of intelligence, even if some of those shots are wrong. That illusion keeps users engaged, which means more users and more money.

And that's why LLM hallucinates :P

Re: Why language models hallucinate

#132

Post by pawelmurias »

utyop22 wrote: post #116
rhubarbtree wrote: post #105

I find this rather oddly phrased. LLMs hallucinate because they are language models. They are stochastic models of language. They model language, not truth. If the “truthy” responses are common in their training set for a given prompt, you might be more likely to get something useful as output. Feels like we fell into that idea and said - ok this is useful as an information retrieval tool. And now we use RL to reinfo…

The reality is, language itself does not capture the entirety of what is really going on. And I'd get argue its the poorest way of expressing - but one that enables transmission through various mediums efficiently on a cost basis. E.g. when I explain a concept, what comes to my mind is not a string of letters and words. There is a mix of imagery and even sounds that I may have acquired from learning about a concept -…

I would assume most people use native subtitles when it's hard to understand what words the actors said.

Re: Why language models hallucinate

#133

Post by rubatuga »

didibus wrote: post #125
rhubarbtree wrote: post #105

I find this rather oddly phrased. LLMs hallucinate because they are language models. They are stochastic models of language. They model language, not truth. If the “truthy” responses are common in their training set for a given prompt, you might be more likely to get something useful as output. Feels like we fell into that idea and said - ok this is useful as an information retrieval tool. And now we use RL to reinfo…

I agree with everything you said except: > Trying to eliminate cases where a stochastic model the size of an LLM gives “undesirable” or “untrue” responses seems rather odd. Take it back to what it is like you say, this is a predictive model, and the work of any ML scientist is to iterate on the model to try and get perfect accuracy on unseen data. It makes sense to want to tune the models to lower the rate of predict…

But we're getting into the limits of knowledge and what is true/untrue. A stochastic model will be wrong sometimes.

Re: Why language models hallucinate

#134

Post by skybrian »

robertclaus wrote: post #119

While I get the academic perspective of sharing these insights, this article comes across as corporate justifying/complaining that their model's score is lower than it should be on the leaderboards... by saying the leaderboards are wrong. Or an even darker take is that its coorporate saying they won't prioritize eliminating hallucinations until the leaderboards reward it.

Yes, it's self-interested because they want to improve the leaderboards, which will help GPT-5 scores, but in the other hand, the changes they suggest seem very reasonable and will hopefully help everyone in the industry do better.

And I'm sure other people will complain if notice that changing the benchmarks makes things worse.

Re: Why language models hallucinate

#135

Post by utyop22 »

pawelmurias wrote: post #132
utyop22 wrote: post #116

Earlier quoted context omitted.

The reality is, language itself does not capture the entirety of what is really going on. And I'd get argue its the poorest way of expressing - but one that enables transmission through various mediums efficiently on a cost basis. E.g. when I explain a concept, what comes to my mind is not a string of letters and words. There is a mix of imagery and even sounds that I may have acquired from learning about a concept -…

I would assume most people use native subtitles when it's hard to understand what words the actors said.

No that is not the reason.

People watch Netflix to switch their brain off - having the text there helps along with the visual and sound to deliver the content. However, text is inferior to both visual and sound as a delivery mechanism.

Re: Why language models hallucinate

#136

Post by jibal »

pawelmurias wrote: post #132
utyop22 wrote: post #116

Earlier quoted context omitted.

The reality is, language itself does not capture the entirety of what is really going on. And I'd get argue its the poorest way of expressing - but one that enables transmission through various mediums efficiently on a cost basis. E.g. when I explain a concept, what comes to my mind is not a string of letters and words. There is a mix of imagery and even sounds that I may have acquired from learning about a concept -…

I would assume most people use native subtitles when it's hard to understand what words the actors said.

That's why I do.

Re: Why language models hallucinate

#137

Post by the_af »

Some people here in the comments are arguing that the LLM "understands" what is "true" and "false", that is somewhat capable of reasoning, etc, but I still find it quite easy (with GPT-5) to break its facade of "reasoning".

I asked it to play a word game. This is very simple, and a very short session too. It failed in its very first response, and then it failed in explaining why it failed. All with total confidence, no hesitation.

Nobody fluent in English would fail so catastrophically. I actually expected it to succeed:

https://chatgpt.com/share/68bcb490-a5b4-8013-b2be-35d27962ad...

It's clear by this failure model the LLM doesn't understand anything.

Edit: to be clear, as the session goes longer it becomes more interesting, but you can still trip the LLM up in ways no human "understanding" the game would. My 6-year old plays this game better, because she truly understands... she can trip up, but not like this.

Re: Why language models hallucinate

#138

Post by didibus »

Jensson wrote: post #123
didibus wrote: post #111

Earlier quoted context omitted.

The word "hallucination" mis-characterizes it. LLMs predict the likely tokens to follow the context. And they can make incorrect predictions. LLMs therefore don't have perfect accuracy of prediction. When their predictions are incorrect, people say they "hallucinate". Nobody questions why predictive weather models aren't perfectly accurate, because it makes sense that a prediction can be wrong. Marketing and hype has…

A weather model is made to predict the weather and used to predict the weather, so there you are right. A language model is made to predict language, but used to generate code or answers to math questions, that is not the same situation as a weather model. The language model is not made to solve math or generate correct code, if you ask it to predict the weather it wont try to predict the weather, it will just predic…

I agree the model is predicting language and not actually running the math. That is a point I try to stress too. It is not thinking through a problem, it is predicting what text would look like if someone were working it out.

But the training does not just reinforce plausible continuations, it biases toward text that matches correct answers. So in that sense they are training it not just to predict any likely text, but to predict text that is more likely to contain the right answer to a math or coding problem.

To me that does not look so different from other ML models. They all work by turning a problem into something a computer can handle statistically, and they all face the same trade offs. Prediction errors are inevitable, and you still have to decide whether to tune for recall, which gives hallucinations, or precision, which gives refusals.

Re: Why language models hallucinate

#139

Post by Difwif »

didibus wrote: post #131

When tuning predictive models you always have to balance precision and recall because 100% accuracy is never going to happen. In LLMs that balance shows up as how often the model hallucinates versus how often it says it doesn’t know. If you push toward precision you end up with a model that constantly refuses: What’s the X of Y? I don’t know. Can you implement a function that does K? I don’t know how. What could be t…

It would be interesting to see two versions of a model. A primary model tuned for precision that's focused on correctness that works with or orchestrates a creative model that's tuned for generating new (and potentially incorrect) ideas. The primary model is responsible for evaluating and reasoning about the ideas/hallucinations. Feels like a left/right brain architecture (even though that's an antiquated model of human brain hemispheres).

Re: Why language models hallucinate

#140

Post by patrickmay »

ACCount37 wrote: post #52
farceSpherule wrote: post #45

I wish they would come up with a better term. Computers do not have brains or conscientiousness. They erroneously construct responses (i.e., confabulation).

You should anthropomorphize LLMs more. Anthropomorphizing LLMs is at least directionally correct 9 times out of 10. LLMs, in a very real way, have "conscientiousness". As in: it's a property that can be measured and affected by training, and also the kind of abstract concept that an LLM can recognize and operate off. If you can just train an LLM to be "more evil", you can almost certainly train an LLM to be "more con…

> You should anthropomorphize LLMs more.

No, you shouldn't. They hate that.

Post reply on HN