Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI hallucinations: Why LLMs make things up (and how to fix it)

kapa.ai

Post reply on HN
Print view View original

111–120 of 257 posts

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#111

Post by namaria »

mdp2021 wrote: post #108
jojobas wrote: post #79

Earlier quoted context omitted.

There is no source of truth for dressage competition results, these are accepted as jury preference judgement. There are plenty of matters where there is such a source of truth, and LLMs don't know the difference.

> There is no source of truth There is no «source of [_simple_] truth» for complex things, but there are more (instead of less) objective complex evaluations. Note that this is also valid for factual notions: e.g., "When were the Pyramids built?".

Ancient Egypt chronology is a poor example of determined knowledge.

We do not know in fact exactly when (which?) Pyramids were built, there are large margin of errors in the estimates.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#112

Post by namaria »

JKCalhoun wrote: post #76
tengbretson wrote: post #73

Earlier quoted context omitted.

LLMs don't know things, they just string together responses that are a best fit for what follows from their prompt. I suspect its so hard to get them to say "I don't know" because if they were biased towards responding that way then I would assume thats almost all they would ever say, since "I don't know" is an appropriate answer to every question imaginable.

I get that, but since it is all probabilities, you might imagine even the LLM knows when it is skating on thin ice. If I'm beginning with "Once / upon / a" I think the data will show a very high confidence in the word to follow with. So too I would imagine it would know when the trail of breadcrumbs it has been following is of the trashier and low probability kind. So just tell me. (Or perhaps speak to me and when yo…

> the LLM knows

I don't think you get it.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#113

Post by namaria »

zmgsabst wrote: post #44
lolinder wrote: post #8

> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…

A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis. In humans. But also apparently in LLMs.

> A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis.

> In humans.

True, but distinguishing reality from imagination is a cornerstone of mental health. And it's becoming apparent that the average person will take the confident spurious affirmations of LLMs as facts, which should call their mental health into question.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#115

Post by namaria »

goatlover wrote: post #109
maeil wrote: post #100

Earlier quoted context omitted.

> At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers. This is the big one missed by the frequent comments on here wondering whether LLMs are a fad, or claiming in their current state they cannot be used to replace…

The bigger, more controversial claim is that LLMs will be net loss for human jobs, when all past automation has been a net positive. Including IT, where automation has led to a vast growth of software jobs, as more can be accomplished with higher level languages, tools, frameworks, etc. For example, compilers didn't put programmers out of business in the 60s, it made programming more available to people with higher l…

A net positive in the long term matters little when it can mean a lifetime of unemployment to a generation of humans. It's easy to dismiss the human suffering incurred during industrialization when we can enjoy its fruits but those who suffered are long dead.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#116

Post by sugarkjube »

ryanmcbride wrote: post #5
Loughla wrote: post #4

I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…

My go-to to show people who don't understand its limitations used to be the old "how many Ms are there in the word 'minimum' or something along those lines, but looks like it's gotten a bit better at that. I just tried it with GPT4o and it gave me the right number, but the wrong placement. In the past it's given it completely wrong: >how many instances of the letter L are in the word parallel The word parallel contai…

Shows nicely what's going on.

If you ask a human, they will answer 3. Sometimes they say 4. Or 2. That's it.

An LLM produces a text using an example it was trained on. They were trained with these elaborate responses, so that's what they produce.

Whenever chatgpt gets something wrong, someone at openai will analyse it, create a few correct examples, and put these on the pile for retraining. Thats why it gets better - not because it is smarter, but it's retrained on your specific test cases.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#117

Post by mdp2021 »

mountainriver wrote: post #12
PLenz wrote: post #11

Everything an LLM returns is an hallucination, it's just that some of those hallucinations line up with reality

The same is true for humans

It's a good thing that, as you state, "humans hold "togetherness" as a "true" value".

But in this context, the value is in how much they have pondered to actually see and evaluate what is eventually seen as "true".

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#118

Post by namaria »

empath75 wrote: post #25
Loughla wrote: post #4

I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…

I refuse to believe that you did any of this with any of the latest models. Gemini and Chat GPT with search are both perfectly capable of producing decent essays with accurate citations. And the 4o model is extremely good at writing python code that can accurately solve math and logic problems. I asked 4o with search to write an essay about the dangers of smoking, along with citations and quotes from the relevant sou…

You exemplify well a big problem with LLMs: When people see accurate enough output on some test question and take it as evidence that they can trust the output to any extent on areas they don't dominate.

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#119

Post by mdp2021 »

namaria wrote: post #111
mdp2021 wrote: post #108

Earlier quoted context omitted.

> There is no source of truth There is no «source of [_simple_] truth» for complex things, but there are more (instead of less) objective complex evaluations. Note that this is also valid for factual notions: e.g., "When were the Pyramids built?".

Ancient Egypt chronology is a poor example of determined knowledge. We do not know in fact exactly when (which?) Pyramids were built, there are large margin of errors in the estimates.

That was my point: answering that question is a more complex evaluation than others. In lower percentiles you may have "what is in front of you" and in upper percentiles you may have "how to fix the balance of power in the Pacific" - all more or less complex evaluations.

I said, "Not even factual notions are trivial, e.g. "When has this event happened" - all have some foundational ground of higher or lower solidity".

Re: AI hallucinations: Why LLMs make things up (and how to fix it)

#120

Post by namaria »

__MatrixMan__ wrote: post #24
PLenz wrote: post #11

Everything an LLM returns is an hallucination, it's just that some of those hallucinations line up with reality

There's room for splitting hairs in there though. Even fiction, for instance, can succeed or fail at being internally consistent, is or is not grammatically correct... Calling everything an AI does a hallucination isn't incorrect, but it reduces the term to meaninglessness. I'm not sure that's most useful thing we can be doing. Atoms are not indivisible, yet we use the term because it works. I anticipate hallucinatio…

> Atoms are not indivisible

They are the smallest unit of a substance that cannot be broken down into smaller units of the same substance. They are, in a sense, indivisible.

Post reply on HN