Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why language models hallucinate

openai.com

Post reply on HN
Print view View original

171–180 of 242 posts

Re: Why language models hallucinate

#171

Post by jibal »

ekianjo wrote: post #154
pawelmurias wrote: post #132

Earlier quoted context omitted.

I would assume most people use native subtitles when it's hard to understand what words the actors said.

Yeah because modern filmmakers make it very hard to hear dialogs for some reason and actors are encouraged to mumble. If I remember correctly even Nolan admitted it.

And they often speak very quickly--I often rewind to catch critical plot points. It's a lot different from a stage play, where actors enunciate so clearly. (Not that I want stage cadence and booming voices from a film ... they are different art forms.)

Also I watch of English language material that uses accents quite different from what my ears are tuned to.

Re: Why language models hallucinate

#172

Post by zahlman »

robotcapital wrote: post #170

It’s interesting that most of the comments here read like projections of folk-psych intuitions. LLMs hallucinate because they “think” wrong, or lack self-awareness, or should just refuse. But none of that reflects how these systems actually work. This is a paper from a team working at the state of the art, trying to explain one of the biggest open challenges in LLMs, and instead of engaging with the mechanisms and ev…

Calling it a "hallucination" is anthropomorphizing too much in the first place, so....

Re: Why language models hallucinate

#173

Post by TychoCelchuuu »

fumeux_fume wrote: post #28

I like that OpenAI is drawing a clear line on what “hallucination” means, giving examples, and showing practical steps for addressing them. The post isn’t groundbreaking, but it helps set the tone for how we talk about hallucinations. What bothers me about the hot takes is the claim that “all models do is hallucinate.” That collapses the distinction entirely. Yes, models are just predicting the next token—but that do…

[dead]

Re: Why language models hallucinate

#174

Post by robotcapital »

zahlman wrote: post #172
robotcapital wrote: post #170

It’s interesting that most of the comments here read like projections of folk-psych intuitions. LLMs hallucinate because they “think” wrong, or lack self-awareness, or should just refuse. But none of that reflects how these systems actually work. This is a paper from a team working at the state of the art, trying to explain one of the biggest open challenges in LLMs, and instead of engaging with the mechanisms and ev…

Calling it a "hallucination" is anthropomorphizing too much in the first place, so....

Right, that’s kind of my point. We call it “hallucination” because we don’t understand it, but need a shorthand to convey the concept. Here’s a paper trying to demystify it so maybe we don’t need to make up anthropomorphized theories.

Re: Why language models hallucinate

#175

Post by BobbyTables2 »

crystal_revenge wrote: post #150
rhubarbtree wrote: post #105

I find this rather oddly phrased. LLMs hallucinate because they are language models. They are stochastic models of language. They model language, not truth. If the “truthy” responses are common in their training set for a given prompt, you might be more likely to get something useful as output. Feels like we fell into that idea and said - ok this is useful as an information retrieval tool. And now we use RL to reinfo…

People also tend not to understand the absurdity of assuming that we can make LLMs stop hallucinating. It would imply not only that truth is absolutely objective , but that it exists on some smooth manifold which language can be mapped to. That means there would be some high dimensional surface representing "all true things". Any fact could be trivially resolved as "true" or "false" simply by exploring whether or not…

Agree. I deeply suspect the problem of asking an LLM to not hallucinate is equivalent to the classic Halting Problem.

Re: Why language models hallucinate

#176

Post by razzmatazmania »

zahlman wrote: post #172
robotcapital wrote: post #170

It’s interesting that most of the comments here read like projections of folk-psych intuitions. LLMs hallucinate because they “think” wrong, or lack self-awareness, or should just refuse. But none of that reflects how these systems actually work. This is a paper from a team working at the state of the art, trying to explain one of the biggest open challenges in LLMs, and instead of engaging with the mechanisms and ev…

Calling it a "hallucination" is anthropomorphizing too much in the first place, so....

Confabulation is human behavioral phenomena that is not all that uncommon. Have you ever heard a grandpa big fish story? Have you ever pretended to know something you didn't because you wanted approval or to feel confident? Have you answered a test question wrong when you thought you were right? What I find fascinating about these models is they are already more intelligent and reliable than the worst humans. I've known plenty of people who struggle to conceptualize and connect information and are helpless outside of dealing with familiar series of facts or narratives. That these models aren't even as large as human brains makes me suspect that practical hardware limits might still be in play here.

Re: Why language models hallucinate

#177

Post by vrighter »

fumeux_fume wrote: post #28

I like that OpenAI is drawing a clear line on what “hallucination” means, giving examples, and showing practical steps for addressing them. The post isn’t groundbreaking, but it helps set the tone for how we talk about hallucinations. What bothers me about the hot takes is the claim that “all models do is hallucinate.” That collapses the distinction entirely. Yes, models are just predicting the next token—but that do…

if you insist that they are different, then please find one logical, non-subjective, way to distinguish between a hallucination and not-a-hallucination. Looking at the output and deciding "this is clearly wrong" does not count. No vibes.

Re: Why language models hallucinate

#178

Post by ninetyninenine »

kdnvk wrote: post #104
juancn wrote: post #103

This is fluff, hallucinations are not avoidable with current models since those are part of the latent space defined by the model and the way we explore it, you'll always find some. Inference is kinda like doing energy minimization on a high dimensional space, the hallucination is already there, for some inputs you're bound to find them.

Did you read the linked paper?

The majority of people on this thread didn't even click on the link. People are so taken by their own metaphysical speculations of what an LLM is.

Like literally the inventor of the LLM wrote an article and everyone is criticizing that article without even reading it. Most of these people have never built an LLM before either.

Re: Why language models hallucinate

#179

Post by ninetyninenine »

ACCount37 wrote: post #8
roxolotl wrote: post #4

This seems inherently false to me. Or at least partly false. It’s reasonable to say LLMs hallucinate because they aren’t trained to say they don’t have a statistically significant answer. But there is no knowledge of correct vs incorrect in these systems. It’s all statistics so what OpenAI is describing sounds like a reasonable way to reduce hallucinations but not a way to eliminate them nor the root cause.

Is there any knowledge of "correct vs incorrect" inside you? If "no", then clearly, you can hit general intelligence without that. And if "yes", then I see no reason why an LLM can't have that knowledge crammed inside it too. Would it be perfect? Hahahaha no. But I see no reason why "good enough" could not be attained.

I'm going to tell you straight up. I am a very intelligent man and I've been programming for a very long time. My identity is tied up with this concept that I am intelligent and I'm a great programmer so I'm not going to let some AI do my job for me. Anything that I can grasp to criticize the LLM I'm gonna do it because this is paramount to me maintaining my identity. So you and your rationality aren't going to make me budge. LLMs are stochastic parrots and EVERYONE on this thread agrees with me. They will never take over my job!

I will add they will never take over my job because it makes me sound more rational and it's easier to swallow that then to swallow the possibility that they will make me irrelevant once the hallucination problem is solved.

Post reply on HN