Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

481–490 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#481

Post by ninetyninenine »

mannykannot wrote: post #480
ninetyninenine wrote: post #473

Earlier quoted context omitted.

>I had no intention of implying any malfeasance in my use of the word "odd"; I mean it in the sense of unusual, unexpected and surprising. The thing is, you finishished your precursor post saying, about your tests and mine, that it comes down to there being a human in the loop making a judgement call, but in a follow-on you say that there are thousands of quantitative metrics. Why, I wondered, would that matter, if i…

At this point, I think it is worth refreshing what the issue here is, which is whether LLMs understand that the language they receive is about an external world, which operates through causes which have nothing to do with token-combination statistics of the language itself. > It matters because of humans... I'm still a bit puzzled here, because it seems to me that the paragraph continuing from here is making the argu…

>I'm still a bit puzzled here, because it seems to me that the paragraph continuing from here is making the argument that LLM performance on these tests doesn't matter, as far as the question is concerned: in this paragraph you seem to be saying (paraphrased) that despite LLMs' impressive performance on these quantitative tests, they could still fail Turing tests, so their performance on these quantitative tests is not decisive.

It matters in the quantitative sense. It measures AI performance. What it won't do is matter to YOU. Because you're a human and humans will keep moving the bar to a higher standard right? When AI shot passed the turing test humans just moved the goal posts. So to convince someone like YOU we have to look at the final metric. The point where LLM I/O becomes indistinguishable/superior to humans. Of course you look at the last decade... AI is rapidly approaching that final bar.

>The impression I get from what you have written in this post is that you are not claiming that a test conforming to your requirements has actually been successfully performed, you are just assuming it could be?

Whether I assume or don't assume, the projection of the trendline currently indicates that it will. Given the trendline that is the most probable conclusion.

>The experiment does not explicitly address it.

Nothing on the face of the earth can address the question. Because nobody truly knows what "understanding" something actually is. You can't even articulate the definition in a formal way such that it can be dictated on a computer program.

So I went to the next best possibility, which is my point. The point is ALTHOUGH we don't know what understanding is, we ALL assume humans understand things. So we set that as a bar metric. Anything indistinguishable from a human must understand things. Anything that appears close to a human but is not quite human must understand things ALMOST as well as a human.

Re: Hallucination is inevitable: An innate limitation of large language models

#482

Post by caditinpiscinam »

username332211 wrote: post #314
caditinpiscinam wrote: post #229

Earlier quoted context omitted.

> Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velo…

The thing is, the usefulness of a question answering system is in answering questions people don't generally know. We don't need an answering system for things that are common knowledge. And it's not uncommon that certain knowledge would be, well uncommon even among experts. Experts specialize. Since the usefulness of ornithological examples is getting exhausted, let's say one out of a hundred lawyers works in bankru…

I agree that without deeper reasoning skills it's not obvious how a model should handle this sort of situation. In the end, I'm not saying that fixing the hallucination issue is easy, but rather that hallucination might be better understood as a property of the training data everyone is using, rather than the models themselves.

Re: Hallucination is inevitable: An innate limitation of large language models

#483

Post by mannykannot »

ninetyninenine wrote: post #481
mannykannot wrote: post #480

Earlier quoted context omitted.

At this point, I think it is worth refreshing what the issue here is, which is whether LLMs understand that the language they receive is about an external world, which operates through causes which have nothing to do with token-combination statistics of the language itself. > It matters because of humans... I'm still a bit puzzled here, because it seems to me that the paragraph continuing from here is making the argu…

>I'm still a bit puzzled here, because it seems to me that the paragraph continuing from here is making the argument that LLM performance on these tests doesn't matter, as far as the question is concerned: in this paragraph you seem to be saying (paraphrased) that despite LLMs' impressive performance on these quantitative tests, they could still fail Turing tests, so their performance on these quantitative tests is n…

> What it won't do is matter to YOU. Because you're a human and humans will keep moving the bar to a higher standard right? When AI shot passed the turing test humans just moved the goal posts. So to convince someone like YOU we have to look at the final metric.

It is disappointing to see you descending into something of a rant here. If you knew me better, you would know that I spend more time debating in opposition to people who think they can prove that AGI/artificial consciousness is impossible than I do with people who think it is already an undeniable fact that it has already been achieved (though this discussion is shifting the balance towards the middle, if only briefly.) Just because I approach arguments in either direction with a degree of skepticism and I don't see any value in trying to call the arrival of true AGI at the very first moment it occurs, it does not mean that I'm trying (whether secretly or openly) to deny that it is possible either in the near-term or at all. FWIW, I regard the former as possible and the latter highly probable, so long as we don't self-destruct first.

> Nothing on the face of the earth can address the question. Because nobody truly knows what "understanding" something actually is. You can't even articulate the definition in a formal way such that it can be dictated on a computer program.

The anti-AI folk I mentioned above would willingly embrace this position! They would say that it shows that human-like intelligence and consciousness lies outside of the scope of the physical sciences, and that this creates the possibility of a type of p-zombie that is indistinguishable by physical science from a human and yet lacks any concept of itself as an entity within an external world.

More relevantly, your response here repeats an earlier fallacy. In practice, concepts and their definitions are revised, tightened, remixed and refined as we inquire into them and gain knowledge. I know you don't agree, but as this is not an opinion but an empirical observation, validated by many cases in the history of science and science-like disciplines, I don't see you prevailing here - and there's the knowledge-bootstrap problem if this were not the case, as well.

It occurred to me this morning that there's a variant or extension of the quantitative Turing test which goes like this:

We have two agents and a judge. The judge is a human and the agents are either a pair of humans, a pair of AIs, or one of each, chosen randomly and without the judge being unaware of the mix. One of the agents is picked, by random choice, to start a discussion with the other with the intent of exploring what the other understands about some topic, with the discussion-starter being given the freedom to choose the topic. The discussion proceeds for a reasonable length of time - let's say one hour.

The judge follows the discussion but does not participate in it. At the conclusion of the discussion, the judge is required to say, for each agent, whether it is more likely that it is a human or AI, and the accuracy of this call is used to assign a categorical variable to the result, just as in the version of the Turing test you have described.

This seems just as quantitative, and in the same way, as your version, yet there's no reason to believe it will necessarily yield the same results. More tests are better, so what's not to like?

Re: Hallucination is inevitable: An innate limitation of large language models

#484

Post by ninetyninenine »

mannykannot wrote: post #483
ninetyninenine wrote: post #481

Earlier quoted context omitted.

>I'm still a bit puzzled here, because it seems to me that the paragraph continuing from here is making the argument that LLM performance on these tests doesn't matter, as far as the question is concerned: in this paragraph you seem to be saying (paraphrased) that despite LLMs' impressive performance on these quantitative tests, they could still fail Turing tests, so their performance on these quantitative tests is n…

> What it won't do is matter to YOU. Because you're a human and humans will keep moving the bar to a higher standard right? When AI shot passed the turing test humans just moved the goal posts. So to convince someone like YOU we have to look at the final metric. It is disappointing to see you descending into something of a rant here. If you knew me better, you would know that I spend more time debating in opposition…

>It is disappointing to see you descending into something of a rant here.

I'm going to be frank with you. I'm not ranting and uncharitable comments like this aren't appreciated. I'm going to respond to your reply later in another post, but if I see more stuff like this I'll stop stop communicating with you. Please don't say stuff like that.

Re: Hallucination is inevitable: An innate limitation of large language models

#485

Post by mannykannot »

ninetyninenine wrote: post #484
mannykannot wrote: post #483

Earlier quoted context omitted.

> What it won't do is matter to YOU. Because you're a human and humans will keep moving the bar to a higher standard right? When AI shot passed the turing test humans just moved the goal posts. So to convince someone like YOU we have to look at the final metric. It is disappointing to see you descending into something of a rant here. If you knew me better, you would know that I spend more time debating in opposition…

>It is disappointing to see you descending into something of a rant here. I'm going to be frank with you. I'm not ranting and uncharitable comments like this aren't appreciated. I'm going to respond to your reply later in another post, but if I see more stuff like this I'll stop stop communicating with you. Please don't say stuff like that.

I could have, equally reasonably, made exactly the same response to your post. I will do my best to respond civilly (I admit that I have some failings in this regard), but I also suggest that whenever you feel the urge to capitalize the word "you", you give it a second thought.

Re: Hallucination is inevitable: An innate limitation of large language models

#486

Post by ninetyninenine »

mannykannot wrote: post #485
ninetyninenine wrote: post #484

Earlier quoted context omitted.

>It is disappointing to see you descending into something of a rant here. I'm going to be frank with you. I'm not ranting and uncharitable comments like this aren't appreciated. I'm going to respond to your reply later in another post, but if I see more stuff like this I'll stop stop communicating with you. Please don't say stuff like that.

I could have, equally reasonably, made exactly the same response to your post. I will do my best to respond civilly (I admit that I have some failings in this regard), but I also suggest that whenever you feel the urge to capitalize the word "you", you give it a second thought.

Apologies, by YOU I mean YOU as a human, not YOU as an individual. Like we all generally feel that the quantitative tests aren't enough. The capitalization was for emphasis for you to look at yourself and know that you're human and likely feel the same thing. Most people would say the stuff like IQ tests aren't enough and we can't pinpoint definitively why, as humans, WE (keyword change) just feel that way.

That feeling is what sets the bar. There's no rhyme or reason behind it. But humans are the one who make the judgement call so that's what it has to be.

I will respond more later when I have time.

Re: Hallucination is inevitable: An innate limitation of large language models

#487

Post by grantcas »

mannykannot wrote: post #483
ninetyninenine wrote: post #481

Earlier quoted context omitted.

>I'm still a bit puzzled here, because it seems to me that the paragraph continuing from here is making the argument that LLM performance on these tests doesn't matter, as far as the question is concerned: in this paragraph you seem to be saying (paraphrased) that despite LLMs' impressive performance on these quantitative tests, they could still fail Turing tests, so their performance on these quantitative tests is n…

> What it won't do is matter to YOU. Because you're a human and humans will keep moving the bar to a higher standard right? When AI shot passed the turing test humans just moved the goal posts. So to convince someone like YOU we have to look at the final metric. It is disappointing to see you descending into something of a rant here. If you knew me better, you would know that I spend more time debating in opposition…

[dead]

Re: Hallucination is inevitable: An innate limitation of large language models

#488

Post by mannykannot »

ninetyninenine wrote: post #486
mannykannot wrote: post #485

Earlier quoted context omitted.

I could have, equally reasonably, made exactly the same response to your post. I will do my best to respond civilly (I admit that I have some failings in this regard), but I also suggest that whenever you feel the urge to capitalize the word "you", you give it a second thought.

Apologies, by YOU I mean YOU as a human, not YOU as an individual. Like we all generally feel that the quantitative tests aren't enough. The capitalization was for emphasis for you to look at yourself and know that you're human and likely feel the same thing. Most people would say the stuff like IQ tests aren't enough and we can't pinpoint definitively why, as humans, WE (keyword change) just feel that way. That feel…

No problem! I anticipate the rest of your response with the expectation that it will be challenging and thought-provoking.

Re: Hallucination is inevitable: An innate limitation of large language models

#489

Post by nkrisc »

cultureswitch wrote: post #345
nkrisc wrote: post #131

Earlier quoted context omitted.

But they could, if they needed to. But most people don’t need to, so they don’t keep that information in their brains. I can’t tell you the date of every time I clip my toenails, but if I had to could remember it.

LLMs can remember their sources. It's just additional knowledge, there's nothing special about it. When you ask an LLM to tell you the height of Mount Everest, it clearly has a map of mountains to heights, in some format. Using exactly the same mapping structure, it can remember a source document for the height.

No, it “knows” that one of the tokens that commonly follows is “29,031 ft”.

Re: Hallucination is inevitable: An innate limitation of large language models

#490

Post by ninetyninenine »

mannykannot wrote: post #488
ninetyninenine wrote: post #486

Earlier quoted context omitted.

Apologies, by YOU I mean YOU as a human, not YOU as an individual. Like we all generally feel that the quantitative tests aren't enough. The capitalization was for emphasis for you to look at yourself and know that you're human and likely feel the same thing. Most people would say the stuff like IQ tests aren't enough and we can't pinpoint definitively why, as humans, WE (keyword change) just feel that way. That feel…

No problem! I anticipate the rest of your response with the expectation that it will be challenging and thought-provoking.

For your test I don't see it offering anything new. I see it as the same as my test but just extra complexities. From a statistical point of view I feel it will yield roughly the same results as my test. As long as the judge outputs a binary true or false on whether the entities are humans or ais.

Yes I did say we can't define understanding. But despite the fact that we can't define it we still counter intuitively "know" when something has the capability of understanding. We say all humans have the capability of understanding.

This is the point. The word is undefined yet we can still apply the word and use the word and "know" whether something can understand things.

Thus we classify humans as capable of understanding things without any rhyme or reason. This is fine. But if you take this logic further, that means anything that is indistinguishable from a human must fit into this category.

That was my point. This is the logical limit of how far we can go with an undefined word. To be consistent with our logical application of the word "understanding" we must apply to AI if AI is indistinguishable from humans. If we don't do this then our reasoning is inconsistent. All of this can be done without even having a definition of the word "understanding"

Post reply on HN