Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

171–180 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#171

Post by codr7 »

diputsmonro wrote: post #66
wrs wrote: post #42

Earlier quoted context omitted.

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be trut…

So much truth here, very refreshing to see!

About time too, the sooner we can stop the madness the better, building a society on top of this technology is a movie I'd rather not see.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#172

Post by leptons »

AnimalMuppet wrote: post #120
leptons wrote: post #107

Earlier quoted context omitted.

Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is…

Human readable, makes grammatical sense, and wrong . And no, that's often not subjective.

It's very subjective. An LLM could return statements like "Global warming is real and man-made", and it also could produce a result like "Global warming is a hoax", and it's definitely up to the reader as to whether the LLM is "hallucinating". It doesn't matter how readable or grammatically correct the LLM is, it's still up to the reader to call bullshit, or not.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#173

Post by ninetyninenine »

abernard1 wrote: post #88
ninetyninenine wrote: post #32

Incomplete training data is kind of a pointless thing to measure. Isn’t incomplete data the whole point of learning in general? The reason why we have machine learning is because data was incomplete. If we had complete data we don’t need ml. We just build a function that maps the input to output based off the complete data. Machine learning is about filling in the gaps based off of a prediction. In fact this is what…

> Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could derive functions (networks) as opposed to deliberate construction of algorithms with GOFAI. But "intelligence" with ML as it stands now is not how humans think. Humans do not need millio…

>> Machine learning is about filling in the gaps based off of a prediction.

>I think this is a generous interpretation of network-based ML.

This is False.

The definition of What you actually do with machine learning is Literally filling in the gaps based on prediction. If you can't see this you may not intuitively understand what ML is in actuality doing.

Let's examine ML in it's most simplest form. Linear Regression based off of 2 data points with a single input X and single output Y:

    (0, 0), (3, 3)
With linear regression this produces a model that's equivalent to : y = x

with y = x you've literally filled an entire domain of infinite possible inputs and outputs from -infinite to positive infinite. From two data points I can now output points like (1,1), (2,2),(343245,343245) literally from the model y=x.

The amount of data given by the model is so overwhelmingly huge that basically it's infinite. You feed in random data into the model at speeds of 5 billion numbers per nano second you will NEVER hit an original data point and you will always be creating novel data from the model.

And there's no law that says the linear regression line even has to TOUCH a data point.

ML is simply a more complex form of what I described above with thousands of values for input, thousands of values for output and thousands of datapoints and a different best fit curve (as opposed to a straight line, to fit into the data points). EVEN with thousands of datapoints you know an equation for a best fit curve basically covers a continuous space and thus holds almost an infinite amount of creative data compared with the amount of actual data points.

Make no mistake. All of ML is ALL about novel data output. It is literally pure creativity..... not memory at all. I'm baffled by all these people thinking that ML models are just memorizing and regurgitating.

The problem this paper is talking about is that the outputs are often illusory. Or to circle back to my comment the "predictions" are not accurate.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#174

Post by codr7 »

ahepp wrote: post #101
wrs wrote: post #42

Earlier quoted context omitted.

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does . If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whos…

maybe hallucination is all cognition is, and humans are just really good at it?

It's definitely part of what cognition is, hallucinogens/meditation/etc allows anyone to verify that much.

Intuitively cognition is several systems running in tandem, supervising and cross checking answers, likely iteratively until some threshold is reached.

Wouldn't surprise me if expert/rule systems are up for some kind of comeback; I feel like we need both, tightly integrated.

There's also dreams, and the role they play in awareness, some kind of self reflective work is probably crucial.

That being said, I'm 100% sure there is something in self awareness that is not part of the system and can't be replicated.

I can observe myself from the outside, actions and reactions, thoughts and feelings; which begs the question: who is acting and reacting, thinking and feeling, and what am I if not that?

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#176

Post by calf »

Terr_ wrote: post #131
calf wrote: post #117

Earlier quoted context omitted.

Your argument makes several mistakes. First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER att…

> You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). No they aren't: When you flip a coin, it landing to display heads or tails is "normal". That's no value judgement, it's just a way to characterize what is common in the mechanics. If it landed perfectly on its edge or was snatched out of the air by a hawk, that would not be "normal",…

You just replaced 'normal' with 'common' to do the heavy lifting, the value judgment remains in the threshold you pick.

Whereas OP said that "hallucinations are part of the normal functioning" of the LLM. I contend their definition of hallucination is too weak and reductive, that scientifically we have not actually settled that hallucinations are a given for LLMs, that humans are an example that LLMs are currently inferior - or else how would you make sense of Terence Tao's assessment of gpt01. It is not a simplistic argument of LLMs are garbage in garbage out, therefore they will always hallucinate. OP doesn't even show they read or understood the paper which is about Turing machine arguments, rather OP is using simplistic semantic and statistical arguments to support their position.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#178

Post by Seb-C »

AlexandrB wrote: post #89
leobg wrote: post #38

Isn’t hallucination just the result of speaking out loud the first possible answer to the question you’ve been asked? A human does not do this. First of all, most questions we have been asked before. We have made mistakes in answering them before, and we remember these, so we don’t repeat them. Secondly, we (at least some of us) think before we speak. We have an initial reaction to the question, and before expressing…

> In fact, if you observe your thinking, you might notice that your thought process often takes on different roles and personas. I don't think it's possible to actually observe one's own thinking. A lot of the "eureka" moments one has in the shower, for example, were probably being thought about somewhere in your head but that process is completely hidden from your conscious mind.

Maybe not observe directly, but analyzing and understanding the thinking process is exactly what psychoanalysis is doing actually. But it takes the help of another person to be able to reach those unconscious thoughts.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#179

Post by FloatArtifact »

lolinder wrote: post #17

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is goi…

This is due to an entire field of AI /machine learning leaning into anthropomorphism shaping terminology reinforced by a narrative.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#180

Post by ekianjo »

zdragnar wrote: post #152
TeMPOraL wrote: post #132

Earlier quoted context omitted.

I think you're going too far here. > By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. It's not a "total coincidence". It's the default. Thus, the model's responses aren't "divorced from any concept of truth or reality" - the whole distribution from which those responses are pulled is strongly aligned with reality. (W…

The problem - as defined by how end users understand it - is that the model itself doesn't know the difference, and will proclaim bullshit with the same level of confidence that it does accurate information. That's how you end up with grocery store chatbots recommending mixing ammonia and bleach for a cocktail, or lawyers using chatbots to cite entirely fictional case law before a judge in court. Nothing that comes o…

> is that the model itself doesn't know the difference, and will proclaim bullshit with the same level of confidence

which is a good model for what humans do as well

Post reply on HN