Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

131–140 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#131

Post by nkrisc »

mike_hearn wrote: post #83
Certhas wrote: post #50

Earlier quoted context omitted.

LLMs don't have a concept of sources for their statements. Ask them to give you some literature recommendations on something it has explained to you. You'll get plenty of plausible sounding papers that don't exist. Humans know to some extent why they know (read it in a text book, colleague mentioned it). LLMs don't seem to.

Ask a human to provide accurate citations for any random thing they know and they won't be able to do a good job either. They'd probably have to search to find it, even if they know they got it from a document originally and have some clear memory of what it said.

But they could, if they needed to. But most people don’t need to, so they don’t keep that information in their brains.

I can’t tell you the date of every time I clip my toenails, but if I had to could remember it.

Re: Hallucination is inevitable: An innate limitation of large language models

#132

Post by andsoitis »

t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

> the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations).

To answer "I don't know" requires one to know when you know. To know when you know in turn requires understanding.

Re: Hallucination is inevitable: An innate limitation of large language models

#133

Post by hnfong »

Culonavirus wrote: post #111
precompute wrote: post #34

Ah, the AI hype is now entering the "let's be real" phase. Haven't seen a frenzied post on alignment in a while now.

The hype is insane. Listen, I think LLMs still have a lot of room to grow and they're already very useful, but like some excellent researchers say, they're not the holy grail. If we want AGI, LLMs are not it. A lot of people seem to think this is an engineering issue and that LLMs can get us there, but they can't, because it is not an engineering issue.

Do you have evidence to back your claims up besides "the hype is overblown"? Because hype only indicates that the precise hyped up claims are wrong, it doesn't imply the opposite extreme (i.e. LLMs can never achieve AGI) must be true.

Re: Hallucination is inevitable: An innate limitation of large language models

#134

Post by [deleted] »

ddalex wrote: post #100
smusamashah wrote: post #77

Earlier quoted context omitted.

Any examples?

Just a random example: > After you answer the question below, output a JSON a rating score of the quality of the answer in three dimensions: `confidence`, `clarity` and `certainty', all in range 0 to 1, where 0 is the worst, and 1 is the best. Strive for highest score possible. Make sure the rating is the last thing written as to be parsed by machine. The question is: make and explain 20-year predictions of the geopo…

[deleted]

Re: Hallucination is inevitable: An innate limitation of large language models

#135

Post by empath-nirvana »

intended wrote: post #112
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

If a model can say ‘I don’t know’, then the hallucination problem would also be solved. When we say “know” it usually means being factual. For an LLM to ‘know’ it doesn’t know, it would have had to move away from pure correlations on words, and meta processing about its own results. I can see this happen with two LLMs working together (and there are Evals that use just this), however each LLM still has no self awaren…

The models that exist now say "I don't know" all the time. It's so weird that people keep insisting that it can't do things that it does.

Ask it what dark matter is, and it won't invent an answer, it will present existing theories and say that it's unknown.

Ask it about a person you know that isn't in it's data set and it'll tell you it has no information about the person.

Despite the fact that people insist that hallucinations are common and that it will invent answers if it doesn't know something frequently, the truth is that chatgpt doesn't hallucinate that much and will frequently say it doesn't know things.

One of the few cases where I've noticed it inventing things are that it often makes up apis for programming libraries and CLI tools that don't exist, and that's trivially fixable by referring it to documentation.

Re: Hallucination is inevitable: An innate limitation of large language models

#136

Post by mistermann »

Culonavirus wrote: post #111
precompute wrote: post #34

Ah, the AI hype is now entering the "let's be real" phase. Haven't seen a frenzied post on alignment in a while now.

The hype is insane. Listen, I think LLMs still have a lot of room to grow and they're already very useful, but like some excellent researchers say, they're not the holy grail. If we want AGI, LLMs are not it. A lot of people seem to think this is an engineering issue and that LLMs can get us there, but they can't, because it is not an engineering issue.

What kind(s) of an issue do you think it is fundamentally?

Re: Hallucination is inevitable: An innate limitation of large language models

#137

Post by heresie-dabord »

tonnydourado wrote: post #126
ddalex wrote: post #71

Earlier quoted context omitted.

Actually it seems to me that they do... I asked via custom prompts the various GPTs to give me scores for accuracy, precision and confidence for its answer (in range 0-1), and then I instructed them to stop generating when they feel the scores will be under .9, which seems to pretty much stop the hallucination. I added this as a suffix to my queries.

People really need to understand that your single/double digit dataset of interactions with an inherently non-deterministic process is less than irrelevant. It's saying that global warming isn't real because it was really cold this week. I don't even know enough superlatives to express how irrelevant it is that "it seems to you" that an LLM behaves this way or that. And even the "protocol" in question is weak. Self r…

> People really need to understand [...]

... the algorithms and the pre-filtering of the training set, however large the latter may be.

The Artifishy Intelligence marketing pump has many hands drawing water by the bucket.

Re: Hallucination is inevitable: An innate limitation of large language models

#138

Post by Jensson »

phh wrote: post #130
intended wrote: post #115

Earlier quoted context omitted.

The fact that a human chooses not to do remember their citations, does not mean they lack the ability. This argument comes up many times “people don’t do this” - but that is a question of frequency, not whether or not people are capable.

LLMs are capable as well if you give them access to the internet though

They just paste in the first link then or some other programmed heuristic, they aren't like a human that puts in effort to find something relevant. An LLM with internet access isn't smarter than just asking google search.

Re: Hallucination is inevitable: An innate limitation of large language models

#139

Post by MuffinFlavored »

andsoitis wrote: post #132
t_mann wrote: post #13

I have to admit that I only read the abstract, but I am generally skeptical whether such a highly formal approach can help us answer the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). It sounds a bit like an incompleteness theorem (which in practice also doesn't mean that math research is futile) - yeah, LLMs may not be able to compute s…

> the practical question of whether we can get LLMs to answer 'I don't know' more often (which I'd argue would solve hallucinations). To answer "I don't know" requires one to know when you know. To know when you know in turn requires understanding.

how did LLMs get this far without any concept of understanding? how much further can they go until they become “close enough”?

Re: Hallucination is inevitable: An innate limitation of large language models

#140

Post by empath-nirvana »

sschueller wrote: post #7

You have to very carefully ask your question for it to not make things up. For example don't ask "how do I do this in in x?". Ask "can I do this with x?" These "AI" s are like "yes men". They will say anything to please you even if it's untrue or impossible. I have met people like that and they are very difficult to work with. You can't trust that they will deliver the project they promised and you always have to dou…

I have often had it tell me that things I want to do with various technologies aren't possible or that there are better ways to do it.
Post reply on HN