Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

91–100 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#91

Post by Jensson »

mike_hearn wrote: post #83
Certhas wrote: post #50

Earlier quoted context omitted.

LLMs don't have a concept of sources for their statements. Ask them to give you some literature recommendations on something it has explained to you. You'll get plenty of plausible sounding papers that don't exist. Humans know to some extent why they know (read it in a text book, colleague mentioned it). LLMs don't seem to.

Ask a human to provide accurate citations for any random thing they know and they won't be able to do a good job either. They'd probably have to search to find it, even if they know they got it from a document originally and have some clear memory of what it said.

Yes, humans wont lie to you about it, they will research and come up with sources. Current LLM doesn't do that when asked for sources (unless they invoke a tool), they come back to you with hallucinated links that looks like links it was trained on.

Re: Hallucination is inevitable: An innate limitation of large language models

#92

Post by tudorw »

sschueller wrote: post #7

You have to very carefully ask your question for it to not make things up. For example don't ask "how do I do this in in x?". Ask "can I do this with x?" These "AI" s are like "yes men". They will say anything to please you even if it's untrue or impossible. I have met people like that and they are very difficult to work with. You can't trust that they will deliver the project they promised and you always have to dou…

I've had some success with 'Ask follow up questions where additional clarity is required'. My best experiences start with a much more freeform conversation about what we are going to attempt to do, doing a Q&A first to make sure that both I and the AI are thinking about the same domain and sharing associated nomenclature seems to help.

Re: Hallucination is inevitable: An innate limitation of large language models

#93

Post by dmd »

az09mugen wrote: post #37
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

I think there is no easy way to make an LLM answer "I don't know". For that, it should learn among all the stuff ingested when people effectively don't know. But most people on internet write down irrelevant stuff even when they don't know instead of simply writing "I don't know". That's a very good point.

Consider the extremely common Amazon product question section, where you see Q: Will this product fit my Frobnitz 123? A: I don't know, I ended up buying something else. Q: Does it come with batteries? A: IDK I RETURN IT

Re: Hallucination is inevitable: An innate limitation of large language models

#94

Post by jddj »

macrolime wrote: post #85

They define an LLM as "a probabilistic model of a string that conditions the output at time t based on all the tokens that come before it in the string". I think that definition is wide enough to include human intelligence, so their finding should be equally valid for humans.

Silk silk silk silk silk silk silk.

What do cows drink?

Re: Hallucination is inevitable: An innate limitation of large language models

#95

Post by chrisjj »

somewhereoutth wrote: post #15
chrisjj wrote: post #2

> hallucination is defined as inconsistencies between a computable LLM and a computable ground truth function. That's simply inaccuracy or fabrication. Labelling it hallucination simply panders to the idea these programs are intelligent.

Yes, imagine a pocket calculator that gave a completely wrong answer 10%? of the time, and calling it 'capricious' instead of simply broken.

Indeed. The best solution would be to market it as intelligent ;)

Re: Hallucination is inevitable: An innate limitation of large language models

#96

Post by moooo99 »

macrolime wrote: post #85

They define an LLM as "a probabilistic model of a string that conditions the output at time t based on all the tokens that come before it in the string". I think that definition is wide enough to include human intelligence, so their finding should be equally valid for humans.

> I think that definition is wide enough to include human intelligence, so their finding should be equally valid for humans.

Which is definitely true. Human memory and the ability to correctly recall things we though we remembered is affected by a whole bunch of things and at times very unreliable.

However, human intelligence, unlike LLMs, is not limited to recalling information we once learned. We are also able to do logical reasoning, which seems to improve in LLMs, but is far from being perfect.

Another problem is how different we treat the reliability of information depending on the source, especially based on personal bias. I think that is a huge factor, because in my experience, LLMs tend to quickly fall over and change their opinion based on user input.

Re: Hallucination is inevitable: An innate limitation of large language models

#97

Post by Jensson »

tgv wrote: post #52
gardenhedge wrote: post #38

Earlier quoted context omitted.

I don't think so. I think it's asking it to repeat code it has been trained on

Search for a piece of code you wrote. If it's more than 15 lines and not boilerplate, chances are you won't find it anywhere on the net.

But there are many pieces of code that I've written that you can find in many places on the net, having a tool that can adapt that to your codebase in seconds is useful. It doesn't have to be smart, just pasting in an function and fitting that to your code is useful.

Re: Hallucination is inevitable: An innate limitation of large language models

#98

Post by jprete »

exe34 wrote: post #46
resolutebat wrote: post #28

Earlier quoted context omitted.

Humans do hallucinate, there's lots of literature on how memories are distorted, we see and hear things we want to see and hear, etc. The particular pathology of LLMs is that they're literally incapable of distinguishing facts from hallucinations even in the most mundane circumstances: if a human is asked to summarize the quarterly results of company X, unlike an LLM they're highly unlikely to recite a convincing but…

And yet if you ask a random person at a rally about their favourite cause of the day, they usually spew sound bites that are factually inaccurate, and give all impressions of being as earnest and confident as the LLM making up quarterly results.

I think that case is complicated at best, because a lot of things people say are group identity markers and not statements of truth. People also learn to not say things that make their social group angry with them. And it's difficult to get someone to reason through the truth or falsehood of group identity statements.

Re: Hallucination is inevitable: An innate limitation of large language models

#99

Post by zer00eyz »

carlossouza wrote: post #31

Someone smart once said: If it is good, we call it "creativity." If it is bad, we call it "hallucination." This isn't a bug (or limitation, as the authors say). It's a feature.

> This isn't a bug

If it isn't a bug, it dam well isn't a hallucination, or creativity.

This is a deeply integrated design defect. One that highlights what we're doing (statistically modeling lots of human language)...

Throwing more data against this path isnt going to magically make it wake up and be an AGI. And this problem is NOT going to go away.

The ML community need to back off the hype train. The first step is them not anthropomorphizing their projects.

Re: Hallucination is inevitable: An innate limitation of large language models

#100

Post by ddalex »

smusamashah wrote: post #77
ddalex wrote: post #71

Earlier quoted context omitted.

Actually it seems to me that they do... I asked via custom prompts the various GPTs to give me scores for accuracy, precision and confidence for its answer (in range 0-1), and then I instructed them to stop generating when they feel the scores will be under .9, which seems to pretty much stop the hallucination. I added this as a suffix to my queries.

Any examples?

Just a random example:

> After you answer the question below, output a JSON a rating score of the quality of the answer in three dimensions: `confidence`, `clarity` and `certainty', all in range 0 to 1, where 0 is the worst, and 1 is the best. Strive for highest score possible. Make sure the rating is the last thing written as to be parsed by machine. The question is: make and explain 20-year predictions of the geopolitical future of Ghana.

Post reply on HN