Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs Will Always Hallucinate, and We Need to Live with This

arxiv.org

Post reply on HN
Print view View original

261–270 of 274 posts

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#261

Post by User23 »

mrbungie wrote: post #159
User23 wrote: post #134

Earlier quoted context omitted.

Are you familiar with Searle's work[1] on the subject? It's fun how topical it is here. Anyhow maybe the medium doesn't matter, but the burden of proof for that claim is on you, because it's contrary to experience, intuition, and thought experiment. [1] https://plato.stanford.edu/entries/chinese-room/

Really out-of-ignorance: Is 'proof' the right word here? A more substantial philosophical counter-argument may be needed, but proof sounds weird in these "metaphysical" (for now) discussions.

You raise a valid point. Proof is an overloaded term with differing meanings in metaphysics, math, and law. However in all three cases it's obviously different attempts to grasp at the same ultimate thing: truth.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#263

Post by AnimalMuppet »

leptons wrote: post #259
AnimalMuppet wrote: post #256

Earlier quoted context omitted.

For the first half of what you said: I will note that "wine capital of France" is a completely different claim than "capital of France", even if many of the words are the same. For the rest: I'll just leave this here for everyone else to judge which of us is being the pedant, and which is arguing just to keep arguing. As for the second half: I am almost in agreement with your overall point here. LLMs are plausible te…

>But LLMs are marketed as more than that, and that's the problem. They're marketed by their makers as more than that. The new snake oil, same as the old snake oil. This is no different than any other tech bubble. Nobody paying attention should think otherwise. I don't care how it's marketed, I mean half the US is going to vote for a serial rapist conman thanks to some twisted marketing. People are idiots and are easi…

No, I never said that an LLM would always say "Paris". I said that Paris is the actual correct answer. I don't give LLMs that kind of credit; I'm not sure what I said that made you think that I do.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#264

Post by gus_massa »

NemoNobody wrote: post #96
gus_massa wrote: post #52

Earlier quoted context omitted.

> A human does not do this. You obviously had never asked me anything. (Specialy tech questions while drinking a cup of cofee.) If I had a cent for every wrong answer, I'd be already a millionair.

Why?? To defend AI you used yourself as an example of how we can also be that dumb too. I don't understand. Your example isn't true - what the OP posted is the human condition regarding this particular topic. You, as a human being obviously kno better than to blurt out the first thing that pop into your head - you even have different preset iterations of acceptable things to blurt in certain situations solely to avoi…

Sorry for the long delay...

Sometime when I'm working, I see items in a circle [1] and I just say "Let's apply the Fourier transform." And I have a few similar rules and gut reactions. You can call it brainstorming, brilliant mathematical intuition, stupid pattern matching or lot of years of experience. I'm not sure and I don't care.

Sometimes I fix my idea intermediately because I realize it's wrong. Sometimes some of my coworkers note the error. Sometimes I have to send an email the next day with a retraction. Sometimes the conclusion is wrong but the main idea is correct and I (or someone else) has to fix it [2].

I make a lot of mistakes, but many of my ideas are good enough to get more questions the next week.

Is my reasoning better than LLM? I hope so (for now). I sometimes take more time before replying and shut up. It's an important feature. Perhaps LLM can get a feature to write a paragraph in a buffer, something like

fake LLM> I have a brilliant idea. Let's try to combine X and Y to solve Z. We first try A and then B and the conclusions is C so ... Wait a minute! Oh! It doesn't work :(. Nevermind. Let's delete this paragraph.

And that paragraph is never shown to the user. Is that implemented? Is that a good idea? Perhaps the LLM must tell the idea to another LLM and in both agree show the result to the user.

Is there a Stroop Effect in math https://en.wikipedia.org/wiki/Stroop_effect ? Perhaps to be as good as a human the LLM must run a few parallel instance and then join the results.

---

About hallucinations: I teach Algebra to first year students in the university. We have to explicitly say that with matrices AB is not equal to BA, and in the "hard" course say that det(A+B) is not equal to det(A)+det(B). Are the students hallucinating math properties? (While looking at the draft of some multiple choice midterms, many times I get distracted and use det(2A)=2det(A) :( .)

---

[1] It's more complicated. The system must have a circular symmetry, not just items in a circle for a nice drawing. Also sometimes the system has more symmetry and the Fourier Transform is only the easy first step.

[2] Sometimes I say "It's obviously true for A, B and C." And when one of my coworkers notice an error in a sign I say. "Then , it's obviously false for A, B and C." Sometimes the A, B and C part is better than my sign calculation.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#265

Post by abernard1 »

imoverclocked wrote: post #102
abernard1 wrote: post #88

Earlier quoted context omitted.

> Machine learning is about filling in the gaps based off of a prediction. I think this is a generous interpretation of network-based ML. ML was designed to solve problems. We had lots of data, and we knew large amounts of data could derive functions (networks) as opposed to deliberate construction of algorithms with GOFAI. But "intelligence" with ML as it stands now is not how humans think. Humans do not need millio…

> Humans do not need millions of examples of cats to know what a cat is. We have evolved over time to recognize things in our environment. We also don’t need to be told that snakes are dangerous as many humans have an innate understanding of that. Our training data is partially inherited.

DNA is around ~725 megabytes. There is no "snakes are dangerous" encoded in there. Our training data is instinctual behaviors we recognize from our sensory inputs.

The idea that we're "pre-trained" the way an LLM is (with hundreds of lifetimes of actual sensory experience) is incorrect.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#266

Post by imoverclocked »

abernard1 wrote: post #265
imoverclocked wrote: post #102

Earlier quoted context omitted.

> Humans do not need millions of examples of cats to know what a cat is. We have evolved over time to recognize things in our environment. We also don’t need to be told that snakes are dangerous as many humans have an innate understanding of that. Our training data is partially inherited.

DNA is around ~725 megabytes. There is no "snakes are dangerous" encoded in there. Our training data is instinctual behaviors we recognize from our sensory inputs. The idea that we're "pre-trained" the way an LLM is (with hundreds of lifetimes of actual sensory experience) is incorrect.

> "The results show that the brain has special neural circuits to detect snakes, and this suggests that the neural circuits to detect snakes have been genetically encoded," Nishijo said.

> The monkeys tested in the experiment were reared in a walled colony and neither had previously encountered a real snake.

[1] https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-....

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#267

Post by data_maan »

noobermin wrote: post #208
data_maan wrote: post #200

Just from the way this paper is written (badly, all kinds of LaTeX errors), my belief that something meaningful was proved here, that some nice mathematical theory has been developed, is low. Example: The first 10 pages are meaningless bla

Sorry but you're just wrong. There are issues but the paper is written well enough. The content (whether this is really a novel enough idea) is debateable because anyone could have told you that LLMs aren't going to develop the halting algorithm.

Have you actually read the paper/know how a ML paper should be written?

Here are some of the issues: - section 1.4.3: Can you explain how societal consequences of LLM hallucinations are in any way relevant for a paper that claims in the abstract to use mathematical theories ("computational theory", although that is an error too, they probably mean computability theory)? At best, such a section should be in the appendix, if not a separate paper. - section 1.2.2: What is with the strange subsections 1.2.2.1, 1.2.2.2 that they use for enumeration? - basic LaTeX errors, e.g. page 19, at L=, spacing is all messed up, authors confuse using "So no, I'm afraid you are wrong. The paper violates many of the unspoken rules of how a paper should be written (which can be learned by reading a lot of ML papers, which I guess the authors haven't done) and based on this alone, as it is, wouldn't make it into a mediocre conference, let alone the ICML, ICLR, NeurIPS.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#268

Post by abernard1 »

imoverclocked wrote: post #266
abernard1 wrote: post #265

Earlier quoted context omitted.

DNA is around ~725 megabytes. There is no "snakes are dangerous" encoded in there. Our training data is instinctual behaviors we recognize from our sensory inputs. The idea that we're "pre-trained" the way an LLM is (with hundreds of lifetimes of actual sensory experience) is incorrect.

> "The results show that the brain has special neural circuits to detect snakes, and this suggests that the neural circuits to detect snakes have been genetically encoded," Nishijo said. > The monkeys tested in the experiment were reared in a walled colony and neither had previously encountered a real snake. [1] https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-... .

All this is saying is our neural architecture has specific places that respond to danger and the instinct of fear. Anyone who has seen an MRI knows this is the case. It does not mean actual knowledge of snakes is encoded in our DNA.

Our brains are "trained" on the data they receive during early development. To the degree that evolutionary pressures stored "data," it stored data about how to make our brains (compute) or physiology more effective.

Modern ML tries to shortcut this by making the architecture dumb and the data numerous. If the creators of ML were in charge of a forced evolution, they'd be arguing we need to make DNA 100s of gigabytes and that we needed to store all the memories of our ancestors in it.

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#269

Post by imoverclocked »

abernard1 wrote: post #268
imoverclocked wrote: post #266

Earlier quoted context omitted.

> "The results show that the brain has special neural circuits to detect snakes, and this suggests that the neural circuits to detect snakes have been genetically encoded," Nishijo said. > The monkeys tested in the experiment were reared in a walled colony and neither had previously encountered a real snake. [1] https://www.ucdavis.edu/news/snakes-brain-are-primates-hard-... .

All this is saying is our neural architecture has specific places that respond to danger and the instinct of fear. Anyone who has seen an MRI knows this is the case. It does not mean actual knowledge of snakes is encoded in our DNA. Our brains are "trained" on the data they receive during early development. To the degree that evolutionary pressures stored "data," it stored data about how to make our brains (compute)…

Yes, we are definitely talking about two different processes. Biology is far more complex, nuanced and inscrutable. We don’t understand what all is in our DNA. We do have strong ideas about it.

When it comes down to it, code is data and DNA is code. There are natural pressures to have less DNA so the hundreds of MB of DNA in humans might be argued to be somewhat minimal. If you have ever dealt with piles of handcrafted code that is meant to be small, you’ll likely have seen some form of spaghetti code… which is what I liken DNA to. Instead of it being written with thought and intention, it’s written with predation, pandemics, famine, war etc.

I agree we tend to simplify our artificial networks, largely because we haven’t figured out how to do better yet. The space is wide open and biology has extreme variety in the examples to choose from. Nature “figured out” how to encode information into the very structure of a neural network. The line defining “code” and “data” is thus heavily blurred and any argument about how humans are far superior because of the “reduced number of training examples” is definitely missing the millennia of evolution that created us in the first place.

If we decided to do evolution and self modifying networks then we will likely look for solutions that converge to the smallest possible network. It will be interesting to watch this play out :)

Re: LLMs Will Always Hallucinate, and We Need to Live with This

#270

Post by ahepp »

peterashford wrote: post #260
ahepp wrote: post #101

Earlier quoted context omitted.

maybe hallucination is all cognition is, and humans are just really good at it?

Humans can hallucinate but later determine that what they thought was occurring was not actually real. LLMs can't do that. What you're saying sounds to me rather like what some people are tempted to do on encountering metaphysics: posing questions like "maybe everything is a dream and nothing we experience is real". Which is a logically valid sentence, I guess, but it really is meaningless. The reason we have words l…

I agree. I shouldn't have used the word "hallucinations" since the point of the conversation above my comment was that they are not really hallucinations by any meaningful definition of the word.

My question was more about whether "babbling" with statistically likely tokens can eventually emerge into real cognition. If we add enough neurons to a neural network, will it achieve AGI? or is there some special sauce that is still missing.

Post reply on HN