Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why language models hallucinate

openai.com

Post reply on HN
Print view View original

211–220 of 242 posts

Re: Why language models hallucinate

#211

Post by simianwords »

ninetyninenine wrote: post #179
ACCount37 wrote: post #8

Earlier quoted context omitted.

Is there any knowledge of "correct vs incorrect" inside you? If "no", then clearly, you can hit general intelligence without that. And if "yes", then I see no reason why an LLM can't have that knowledge crammed inside it too. Would it be perfect? Hahahaha no. But I see no reason why "good enough" could not be attained.

I'm going to tell you straight up. I am a very intelligent man and I've been programming for a very long time. My identity is tied up with this concept that I am intelligent and I'm a great programmer so I'm not going to let some AI do my job for me. Anything that I can grasp to criticize the LLM I'm gonna do it because this is paramount to me maintaining my identity. So you and your rationality aren't going to make…

Ha. I could have written this post myself.

Re: Why language models hallucinate

#212

Post by simianwords »

ninetyninenine wrote: post #39
amelius wrote: post #9

They hallucinate because it's an ill-defined problem with two conflicting usecases: 1. If I tell it the first two lines of a story, I want the LLM to complete the story. This requires hallucination, because it has to make up things. The story has to be original. 2. If I ask it a question, I want it to reply with facts. It should not make up stuff. LMs were originally designed for (1) because researchers thought that…

Did you read the article? You’re going on some generic tangent and regurgitating the same spiel about LLMs that you see all over the internet. I mean it’s plain that you have an orthogonal (though generic) opinion on why LLMs hallucinate but how does that relate to the article? How does your opinion which you blatantly just dropped as if it’s the final opinion override the opinion of the article? Seems off topic hone…

I agree. It’s just people who have a different view taking their opportunity to vent out their frustration.

Re: Why language models hallucinate

#213

Post by crabmusket »

codethief wrote: post #199
crabmusket wrote: post #147

Earlier quoted context omitted.

> I don’t think that’s how humans work. Every time this comes up I have to bring up Deutsch. He has the best description of intelligent cognition that I've come across. He takes Popper's "conjecture and criticism" approach to science and argues that this guess-and-check loop applies to all our thinking. E.g. understanding spoken language has some elements of guessing what might have been said and checking that agains…

> Every time this comes up I have to bring up Deutsch. He has the best description of intelligent cognition that I've come across. Would you have a reference?

If you like books, read The Beginning of Infinity. If you don't, I can't help! I wish there were something I could point to online, but nothing really encapsulates the lessons I took from that book. Yes, I'll have to write that thing one day.

Re: Why language models hallucinate

#214

Post by codethief »

crabmusket wrote: post #213
codethief wrote: post #199

Earlier quoted context omitted.

> Every time this comes up I have to bring up Deutsch. He has the best description of intelligent cognition that I've come across. Would you have a reference?

If you like books, read The Beginning of Infinity . If you don't, I can't help! I wish there were something I could point to online, but nothing really encapsulates the lessons I took from that book. Yes, I'll have to write that thing one day.

Thanks so much!

Re: Why language models hallucinate

#215

Post by CGMthrowaway »

aleph_minus_one wrote: post #2

> Think about it like a multiple-choice test. If you do not know the answer but take a wild guess, you might get lucky and be right. Leaving it blank guarantees a zero. In the same way, when models are graded only on accuracy, the percentage of questions they get exactly right, they are encouraged to guess rather than say “I don’t know.” To me, this seems to be an "US-American" way of thinking about multiple-choice t…

>> Think about it like a multiple-choice test. If you do not know the answer but take a wild guess, you might get lucky and be right. Leaving it blank guarantees a zero. In the same way, when models are graded only on accuracy, the percentage of questions they get exactly right, they are encouraged to guess rather than say “I don’t know.”

For TIMED multiple-choice tests (and the timed constraint makes sense in OP analogy as well), probabilistic answering is the kryptonite that lets smart people do well on SATs and IQ tests and other things like that.

I took an IQ test recently and it all came rushing back to me.

For math problems, often the right answer can be found just by inspecting the ones digit of the possible answers and process of elimination. Others, by abstracting what errors the test writer is expecting you to make, and eliminating those as possible answers. It's like magic. Sure, you could actually sit and SOLVE each problem, but when spend the time, when time is valuable?

Pretty sure these types of strategies are not actively taught to anyone unless you have a good college counselor /interested teacher/ SAT tutor. But perhaps they ought to be.

Re: Why language models hallucinate

#216

Post by mannykannot »

throwawaymaths wrote: post #70
mannykannot wrote: post #43

Earlier quoted context omitted.

There is a way to state Parson's point which avoids this issue: hallucinations are just as much a consequence of the LLM working as designed as are correct statements.

fine. which part is the problem?

I suppose you are aware that, for many uses of LLMs, the propensity for hallucinating is a problem (especially if this is not properly taken into account by the people hoping to use these LLMs), but this then leaves me puzzled about what you are asking here.

Re: Why language models hallucinate

#217

Post by vrighter »

esafak wrote: post #181
vrighter wrote: post #177

Earlier quoted context omitted.

if you insist that they are different, then please find one logical, non-subjective, way to distinguish between a hallucination and not-a-hallucination. Looking at the output and deciding "this is clearly wrong" does not count. No vibes.

> Looking at the output and deciding "this is clearly wrong" does not count. You need the ground truth to be able to make that determination, so using your knowledge does count. If you press the model to answer even when it does not know, you get confabulation. What today's models lack is the ability to measure their confidence, so they know when to abstain.

so... vibes. got it. There is no ground truth to compare to in most cases. Because they are not in the training data where you can make objective quantifiable measurements on the statistics.

Re: Why language models hallucinate

#218

Post by player1234 »

fumeux_fume wrote: post #28

I like that OpenAI is drawing a clear line on what “hallucination” means, giving examples, and showing practical steps for addressing them. The post isn’t groundbreaking, but it helps set the tone for how we talk about hallucinations. What bothers me about the hot takes is the claim that “all models do is hallucinate.” That collapses the distinction entirely. Yes, models are just predicting the next token—but that do…

Correct, it is a useless term with the goal to gaslight and antropmorphise a system that predicts the next token.

Re: Why language models hallucinate

#219

Post by player1234 »

esafak wrote: post #181
vrighter wrote: post #177

Earlier quoted context omitted.

if you insist that they are different, then please find one logical, non-subjective, way to distinguish between a hallucination and not-a-hallucination. Looking at the output and deciding "this is clearly wrong" does not count. No vibes.

> Looking at the output and deciding "this is clearly wrong" does not count. You need the ground truth to be able to make that determination, so using your knowledge does count. If you press the model to answer even when it does not know, you get confabulation. What today's models lack is the ability to measure their confidence, so they know when to abstain.

There is no such thing as confidence regarding the actual facts, only confidence in probable output from the input. Factual confidence is impossible with current architecture.

Re: Why language models hallucinate

#220

Post by player1234 »

jdietrich wrote: post #169
crystal_revenge wrote: post #150

Earlier quoted context omitted.

People also tend not to understand the absurdity of assuming that we can make LLMs stop hallucinating. It would imply not only that truth is absolutely objective , but that it exists on some smooth manifold which language can be mapped to. That means there would be some high dimensional surface representing "all true things". Any fact could be trivially resolved as "true" or "false" simply by exploring whether or not…

>People also tend not to understand the absurdity of assuming that we can make LLMs stop hallucinating. It would imply not only that truth is absolutely objective, but that it exists on some smooth manifold which language can be mapped to. Frankly, this is a silly line of argument. There is a vast spectrum between regularly inventing non-existent citations and total omniscience. "We can't define objective truth" isn'…

[flagged]
Post reply on HN