Hallucination is inevitable: An innate limitation of large language models • Hacker News Board

Re: Hallucination is inevitable: An innate limitation of large language models

#221

Post by robrenaud » Sun, Feb 25, 2024, 4:14 PM UTC

p1esk wrote: post #208

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

A smart bullshit artist who loves the sound of their own voice is going to generate more hallucinations than a less smart, more humble, more self aware person.

Making LLMs more knowledgeable is great (more data, bigger models, yay!), but there are other avenues of plausible attack as well. Enabling LLMs to know when they have veered off distribution might work. That is, the LLM doesn't have to know more of the world, it just has to know what it knows and stay there. A person who says "I don't know" is a lot more valuable than an overzealous one who spouts nonsense confidently. Encouraging an LLM to say that there is a disagreement about a topic rather than picking one lane is also a valuable way forward.

Re: Hallucination is inevitable: An innate limitation of large language models

#222

Post by FpUser » Sun, Feb 25, 2024, 4:16 PM UTC

p1esk wrote: post #208

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

The only way to reduce hallucinations in both humans and LLMs is to increase their general intelligence and their knowledge of the world.

You post amounts to: in order to be smarter I need to increase my smartness. Great insight.

Re: Hallucination is inevitable: An innate limitation of large language models

#223

Post by andsoitis » Sun, Feb 25, 2024, 4:21 PM UTC

fl7305 wrote: post #201

sandworm101 wrote: post #163

Earlier quoted context omitted.

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

> a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns That was perhaps true of earlier and smaller LLMs, like GPT-1 and GPT-2. But as they grew larger and were trained with more and more data, they changed from pure pattern matching to implementing algorithms to compress more information into their structure than pure pattern matching can achieve. These algorithms are incompl…

> It will happily execute this algorithm. For large numbers, it is slightly off on the arithmetic. When I asked it to double check, it did so using Python code. After that, it kept using Python code to perform the math. It was also able to reason intelligently about different outcomes if always picking a (or b) given different starting points.

Notice that you had to notice the error and had to prompt it to double check. Lots of complicated things going on here. Many (most?) humans will fail somewhere along this trajectory.

Did it double check the Python code to make sure it is correct (not just in the sense that it is valid, executable code, but that it is the correct check in the first place)? Or did you double check that its modified algorithm is correct? Fool me once and all that…

Upon reflection it appears as if you have a heuristic (algorithm? that leverages logic, awareness, critical thinking, experience, a goal in mind, intuition, etc. to push towards better results.

“It was able to reason intelligently” imbues qualities that I am skeptical is reasonable to attribute to this very narrow domain - what’s an example where it showed intelligent reasoning capabilities?

Re: Hallucination is inevitable: An innate limitation of large language models

#224

Post by foobiekr » Sun, Feb 25, 2024, 4:22 PM UTC

ryanwaggoner wrote: post #165

pooper wrote: post #121

Earlier quoted context omitted.

I changed geopolitical future of Ghana to future of Michael Jackson and Google Bard/Gemini somehow switched to Chinese? ---- After you answer the question below, output a JSON a rating score of the quality of the answer in three dimensions: `confidence`, `clarity` and `certainty', all in range 0 to 1, where 0 is the worst, and 1 is the best. Strive for highest score possible. Make sure the rating is the last thing wr…

I’m really curious how you managed that. I pasted your exact prompt and GPT-3.5 gave me this: === Making 20-year predictions about the future of Michael Jackson is challenging due to his passing in 2009. However, his legacy as a cultural icon and musical genius will likely endure for decades to come. His music will continue to influence future generations, and his impact on pop culture will remain significant. Additi…

GPT-4’s tendencies to write these long winded but mostly empty responses is so frustrating.

Re: Hallucination is inevitable: An innate limitation of large language models

#225

Post by tsimionescu » Sun, Feb 25, 2024, 4:25 PM UTC

tempest_ wrote: post #197

sandworm101 wrote: post #170

Earlier quoted context omitted.

The solution is older non-AI tech. Google search can say "no good results found" because it returns actual data rather than creating anything new. If you want a hard answer about the presence or absence of something, AI isnt the correct tool.

Can, but doesn't. I can't remember the last time google actually returned no results.

It does reply with no results, but only for very long queries. E.g. If you search for two concatenated GUIDs, you can easily see a no results page.

Re: Hallucination is inevitable: An innate limitation of large language models

#226

Post by tsimionescu » Sun, Feb 25, 2024, 4:27 PM UTC

cpuguy83 wrote: post #190

cornholio wrote: post #63

Earlier quoted context omitted.

Transformers have no capacity for self reflection, for reasoning about their reasoning process, they don't "know" that they don't know. My interpretation of the paper is that it claims this weakness if fundamental, you can train the network to act as if it knows its knowledge limits, but there will always be an impossible to cover gap for any real world implementation.

Are humans not the same way? There's a saying "I don't know what I don't know".

The existence of such a saying means little: it is uncontroversially true that humans often do know what they don't know.

For example, I know for sure that I don't know how long I will live. This disproves the saying.

Re: Hallucination is inevitable: An innate limitation of large language models

#227

Post by Fripplebubby » Sun, Feb 25, 2024, 4:27 PM UTC

someplaceguy wrote: post #219

Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

> because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I haven't read the paper, but that sounds like it would only be true if the definition of "hallucinating" is giving a wrong answer, but that's not how it's commonly understood. When people refer to LLMs hallucinating, they are indeed referring to an LLM giving a wrong (and confident) answer. However, not all wrong answers are hallucina…

Yes, I think you're right. I think one way to phrase the authors' argument is:

* There is a class of problems which are harder than polynomial time complexity to solve, but are not np-complete

* LLMs will generate an "answer" in formal language to this class of problems posed to it

* LLMs can at most solve problems with polynomial time complexity due to their fundamental design and principles

* Therefore, LLMs cannot solve > polynomial problems and not np-complete problems either

All of which I buy completely. But I think what people are more interested in is, why is it that the LLM gives an answer when we can prove that it cannot answer this problem correctly? And perhaps that is more related to the commonsense notion of hallucination than I first gave it credit for. Maybe the reason that an LLM gives a formal language answer is the same reason it gives a hallucinatory answer in natural language. But I don't think the paper sheds light on that question

Re: Hallucination is inevitable: An innate limitation of large language models

#228

Post by DiogenesKynikos » Sun, Feb 25, 2024, 4:27 PM UTC

beardedwizard wrote: post #215

DiogenesKynikos wrote: post #200

Earlier quoted context omitted.

> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. Are you sure you're not also describing the human brain? At some point, after we have sufficiently demystified the workings of the human brain, it will probably also sound something like, "Well, the brain is just a large machine that does X, Y and Z [insert banal-soundi…

Human brains form new connections dynamically. Llms are trained on connections human brains have already made. They never make new connections that aren't in training data. Nothing was synthesized, all the data was seen before and related to each other by vector similarity. It can just parrot the collective understanding humans already have and teach it.

> It can just parrot the collective understanding humans already have and teach it.

The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that requires understanding.

> They never make new connections that aren't in training data.

This is just categorically untrue. They make all sorts of logical connections that are not explicitly contained in the training data. Making logical inferences about subjects one has never heard about - based on the things one does know - is an expression of understanding. LLMs do that.

Re: Hallucination is inevitable: An innate limitation of large language models

#229

Post by caditinpiscinam » Sun, Feb 25, 2024, 4:29 PM UTC

username332211 wrote: post #195

caditinpiscinam wrote: post #180

Earlier quoted context omitted.

In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker news and you see a question you don't know the answer to, you normally just don't say anything. If LLMs are being trained on conversations pulled from the internet then they're missing out on a ton of uncertain responses. Maybe LLMs don't truly "understand" q…

If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as ea…

> Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer?

I don't know :)

Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velocity of an unladen swallow, and 99 of them told me "I have no idea" then I'd be pretty skeptical of the one ornithologist who did give me an answer, even if they were very confident.

Re: Hallucination is inevitable: An innate limitation of large language models

#230

Post by elicksaur » Sun, Feb 25, 2024, 4:30 PM UTC

ninetyninenine wrote: post #182

Fiction and story writing is hallucination. It is the opposite of a stochastic parrot. We've achieved both extremes of AI. Computers can be both logical machines and hallucinators. Our goal is to create a machine that can be both at the same time and can differentiate between both. That's the key. Hallucination is important but the key is for the computer to be self aware about when it's hallucinating. Of course it's…

Comparing religion to LLM mistakes is a case of the very prevalent anthropomorphism in society currently. I fear this misunderstanding and conflation will prevent us actually improving the tech.

Coming to an incorrect explanation such as, “Helios pulls the Sun across the sky every day,” is categorically different than a math program incorrectly returning the most likely next token in a sequence. LLMs don’t have beliefs at all.

Helios answers a question “Why does the Sun rise?” Holding such a belief shows a logical understanding that some force must be doing this, but due to a lack of knowledge of the world, the person comes up with an incorrect explanation.

LLMs can’t pose and reason about such questions. It is just not the same class of “hallucinations.” Assuming we’ve figured out cognition via word prediction is going to get us nowhere fast in the long term.