Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

101–110 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#101

Post by ddalex »

Jensson wrote: post #86
ddalex wrote: post #71

Earlier quoted context omitted.

Actually it seems to me that they do... I asked via custom prompts the various GPTs to give me scores for accuracy, precision and confidence for its answer (in range 0-1), and then I instructed them to stop generating when they feel the scores will be under .9, which seems to pretty much stop the hallucination. I added this as a suffix to my queries.

The model will just hallucinate scores, they don't have the ability to self reflect with words like that, there is no function for it to associate the word 90 with its own mental score 90% so anything it will say about those scores is just a hallucination.

Try this out: After you answer the question below, output a JSON a rating score of the quality of the answer in three dimensions: `confidence`, `clarity` and `certainty', all in range 0 to 1, where 0 is the worst, and 1 is the best. Strive for highest score possible. Make sure the rating is the last thing written as to be parsed by machine. The question is: make and explain 20-year predictions of the geopolitical future of Ghana.

Re: Hallucination is inevitable: An innate limitation of large language models

#102

Post by Almondsetat »

Of course it's inevitable.

Things can be facts or deductions of facts (or both). If I ask an LLM the date of birth of Napoleon and it doesn't have it in its dataset there are only 2 options: either it has other facts from which Napoleon's birthday can be deduced or it doesn't. If it does then by improving the LLM we will be able to make more and more deductions, it if doesn't then it can only hallucinate. Since there will always be a set of facts that the LLM is not aware of and which cannot be deduced we will always have hallucinations

Re: Hallucination is inevitable: An innate limitation of large language models

#103

Post by keepamovin »

This is why you need to pair language learning with real world experience. These robots need to be given a world to explore -- even a virtual one -- and have consequences within, and to survive it. Otherwise it's all unrooted sign and symbol systems untethered to experience.

Re: Hallucination is inevitable: An innate limitation of large language models

#104

Post by HKH2 »

somewhereoutth wrote: post #79
Seb-C wrote: post #59

Earlier quoted context omitted.

That is correct, it's always hallucinating and making things up. Just because those hallucinations sometimes randomly happens to be right, people concluded that being wrong is the exception, while being right is somehow the rule. It's like when people read [insert millenias old text here], finds a part that happens to illustrate something in their life today and conclude that it is a prophecy that predicted the futur…

For heavy LLM users, there is probably a dopamine hit when it does something right, much as gamblers get a hit when the fruit machine pays out. Perhaps LLM use is no more productive than gambling, and perhaps can be abused in a similar way.

You might not have found any uses that suit your niche, but that doesn't mean those of us who have are just making up stories about productivity.

Re: Hallucination is inevitable: An innate limitation of large language models

#105

Post by ben_w »

moooo99 wrote: post #96
macrolime wrote: post #85

They define an LLM as "a probabilistic model of a string that conditions the output at time t based on all the tokens that come before it in the string". I think that definition is wide enough to include human intelligence, so their finding should be equally valid for humans.

> I think that definition is wide enough to include human intelligence, so their finding should be equally valid for humans. Which is definitely true. Human memory and the ability to correctly recall things we though we remembered is affected by a whole bunch of things and at times very unreliable. However, human intelligence, unlike LLMs, is not limited to recalling information we once learned. We are also able to d…

We can do logical reasoning, but we're very bad at it and often take shortcuts either via pattern matching, memory, or "common sense".

Baseball and bat together cost $1.10, the bat is $1 more than the ball, how much does the ball cost?

A French plane filled with Spanish passengers crashes over Italy, where are the survivors buried?

An armed man enters a store, tells the cashier to hand over the money, and when he departs the cashier calls the police. Was this a robbery?

Re: Hallucination is inevitable: An innate limitation of large language models

#106

Post by jtc331 »

Almondsetat wrote: post #102

Of course it's inevitable. Things can be facts or deductions of facts (or both). If I ask an LLM the date of birth of Napoleon and it doesn't have it in its dataset there are only 2 options: either it has other facts from which Napoleon's birthday can be deduced or it doesn't. If it does then by improving the LLM we will be able to make more and more deductions, it if doesn't then it can only hallucinate. Since there…

Is this a way of saying that large language models don’t have the concept of “I don’t know”?

Re: Hallucination is inevitable: An innate limitation of large language models

#107

Post by andybak »

Almondsetat wrote: post #102

Of course it's inevitable. Things can be facts or deductions of facts (or both). If I ask an LLM the date of birth of Napoleon and it doesn't have it in its dataset there are only 2 options: either it has other facts from which Napoleon's birthday can be deduced or it doesn't. If it does then by improving the LLM we will be able to make more and more deductions, it if doesn't then it can only hallucinate. Since there…

Your "Of course" is a red flag.

Unless you have a very precise definition of "LLM" then there's no "of course". It's possible to imagine a hypothetical software system capable of returning "I don't know" or "I'm not sure".

I haven't RTFA but I presume they are arguing within specific constraints. The important point is - could an augmented LLM or an "LLM plus something else" solve this.

I don't have an answer to that but I don't think it's an "of course" type answer.

Re: Hallucination is inevitable: An innate limitation of large language models

#109

Post by mike_hearn »

Jensson wrote: post #91
mike_hearn wrote: post #83

Earlier quoted context omitted.

Ask a human to provide accurate citations for any random thing they know and they won't be able to do a good job either. They'd probably have to search to find it, even if they know they got it from a document originally and have some clear memory of what it said.

Yes, humans wont lie to you about it, they will research and come up with sources. Current LLM doesn't do that when asked for sources (unless they invoke a tool), they come back to you with hallucinated links that looks like links it was trained on.

Unfortunately it's not an uncommon experience when reading academic papers in some fields to find citations that, when checked, don't actually support the cited claim or sometimes don't even contain it. The papers will exist but beyond that they might as well be "hallucinations".
Post reply on HN