Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Overcoming the limits of current LLMs

seanpedersen.github.io

Post reply on HN
Print view View original

81–90 of 111 posts

Re: Overcoming the limits of current LLMs

#81

Post by adrianmonk »

darby_nine wrote: post #4

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

On a literal level, hallucinations are perceptual.

But "hallucination" was already (before LLMs) being used in a figurative sense, i.e. for abstract ideas that are made up out of nothing. The same is also true of other words that were originally visual, like "illusion" and "mirage".

Re: Overcoming the limits of current LLMs

#82

Post by rtkwe »

sean_pedersen wrote: post #78
dweinus wrote: post #75

Earlier quoted context omitted.

100% I think the author is really misunderstanding the issue here. "Hallucination" is a fundamental aspect of the design of Large Language Models. Narrowing the distribution of the training data will reduce the LLM's ability to generalize, but it won't stop hallucinations.

I agree in that a perfectly consistent dataset won't completely stop statistical language models from hallucinating but it will reduce it. I think it is established that data quality is more important than quantity. Bullshit in -> bullshit out, so a focus on data quality is good and needed IMO. I am also saying LMs output should cite sources and give confidence scores (which reflects how much the output is in or out…

I think the problem is you need an extremely large quantity of data just to get the machine to work in the first place. So much so that there may not be enough to get it working on just "quality" data.

Re: Overcoming the limits of current LLMs

#83

Post by mistermann »

antisthenes wrote: post #76
smokel wrote: post #39

Earlier quoted context omitted.

> We can't develop a universally coherent data set because Yet every child seems to manage, when raised by a small village, over a period of about 18 years. I guess we just need to give these LLMs a little more love and attention.

And then you go out into the real world, talk to real adults, and discover that the majority of people don't have a coherent mental model of the world, and have completely ridiculous ideas that aren't anywhere near an approximation of the real physical world.

> and discover that the majority of people don't have a coherent mental model of the world

"Coherent" is doing a lot of lifting here. All humans have highly flawed models, and we've been culturally conditioned to grade on a curve to hide the problem from ourselves.

Re: Overcoming the limits of current LLMs

#84

Post by Terr_ »

mewpmewp2 wrote: post #37
devjab wrote: post #28

Earlier quoted context omitted.

I disagree with this take because LLMs are, always, hallucinating. When they get things right it’s because they are lucky. Yes, yes, it’s more complicated than that, but the essence of LLMs is that they are very good at being lucky. So good that they will often give you better results than random search engine clicks, but not good enough to be useful for anything important. I think what calling the times they get thi…

But the point is, isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? It is the memory pathways leading them astray. It could be thought of a memory system that at certain point any longer can't be fully sure if whatever connections they have are from actually being trained or it or created accidentally.

> isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself?

I suppose so, in the sense that someone could simply be lying about pink elephants instead of seeing them. However it's hard to argue that the machine knows the "right" answer and is (intelligently?) deceiving us.

> It is the memory pathways leading them astray.

I don't think it's a "memory" issue as much as a "they don't operate the way we like to think they do" issue.

Suppose a human is asked to describe different paintings on the wall of an art gallery. Sometimes their statements appear valid and you nod along, and sometimes the statements are so wrong that it alarms you, because "this person is hallucinating."

Now consider how the entire situation is flipped by finding out one additional fact... They're actually totally blind.

Is it a lie? Is it a hallucination? Does it matter? Either way you must dramatically re-evaluate what their "good" outputs really mean and whether they can be used.

Re: Overcoming the limits of current LLMs

#85

Post by Terr_ »

marcosdumay wrote: post #72
Terr_ wrote: post #32

Earlier quoted context omitted.

To offer a satirical analogy: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: Hindenburg 2 will only contain only normal and unreactive hydrogen gas, and not the rare and unusual explosive kind, which is merely a temporary hurdle in this highly dynamic and growing field." Edit: It retrospect, perhaps a better analogy would involve gasoline, as its explosive n…

Yes (to the edit), an analogy with making planes safer by only using non-flammable fuels is perfect.

I expect most people have already filled in the blanks, but for completeness:

"Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: The Ford Pinto-II will only contain only normal and stable gasoline, and not the rare and unusual burning kind, which is merely a temporary hurdle in this highly dynamic and explos--er--fast growing field."

Re: Overcoming the limits of current LLMs

#86

Post by [deleted] »

lsy wrote: post #31

We can't develop a universally coherent data set because what we understand as "truth" is so intensely contextual that we can't hope to cover the amount of context needed to make the things work how we want, not to mention the numerous social situations where writing factual statements would be awkward or disastrous. Here are a few examples of statements that are not "factual" in the sense of being derivable from a u…

[deleted]

Re: Overcoming the limits of current LLMs

#87

Post by mehrau »

mitthrowaway2 wrote: post #62

LLMs don't only hallucinate because of mistaken statements in their training data. It just comes hand-in-hand with the model's ability to remix, interpolate, and extrapolate answers to other questions that aren't directly answered in the dataset. For example if I ask ChatGPT a legal question, it might cite as precedent a case that doesn't exist at all (but which seems plausible, being interpolated from cases that do…

[dead]

Re: Overcoming the limits of current LLMs

#88

Post by mewpmewp2 »

Terr_ wrote: post #84
mewpmewp2 wrote: post #37

Earlier quoted context omitted.

But the point is, isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? It is the memory pathways leading them astray. It could be thought of a memory system that at certain point any longer can't be fully sure if whatever connections they have are from actually being trained or it or created accidentally.

> isn't hallucinating about having malformed, altered or out of touch input rather than producing inaccurate output yourself? I suppose so, in the sense that someone could simply be lying about pink elephants instead of seeing them. However it's hard to argue that the machine knows the "right" answer and is (intelligently?) deceiving us. > It is the memory pathways leading them astray. I don't think it's a "memory" i…

To me it's more like, imagine that you have read a lot of books throughout your life, but then someone comes in and asks a question from you and you try to answer from memory, but you get beaten when you say something like "I don't know", and you get rewarded if you answer accurately. You do get beaten if you answer inaccurately, but eventually you learn that if you just say something, you might just be accurate and you will not get beaten. So you just always learn to answer to the best of your knowledge, while never saying that you specifically don't know, because it decreases chances of getting beat up. You are not intentionally lying, you are just hoping that whatever you say is accurate to the best you can do according to the neural connections you've built up in your brain.

Like you ask me for a birthdate of some obscure political figure from history? I'm going to try to feel out what period in history the name might feel like to me and just make my best guess based on that, then say some random year and a birthdate. It just has the lowest odds of being beaten. Was I hallucinating? No, I was just trying to not get beaten.

Re: Overcoming the limits of current LLMs

#89

Post by motohagiography »

threeseed wrote: post #46
RodgerTheGreat wrote: post #3

One of the main factors that makes LLMs popular today is that scaling up the models is a simple and (relatively) inexpensive matter of buying compute capacity and scraping together more raw text to train them. Without large and highly diverse training datasets to construct base models, LLMs cannot produce even the superficial appearance of good results. Manually curating "tidy", properly-licensed and verified dataset…

> properly-licensed and verified datasets is immensely more difficult, expensive Arguably the bigger problem is that many of those datasets e.g. WSJ articles are proprietary and can be exclusively licensed like we've seen recently with OpenAI. So we end up with in a situation where competition is simply not possible.

the irony is that if large media providers aren't represented in the training sets, my comments on internet forums over the decades will be over-represented, which is kind of great, really.

Re: Overcoming the limits of current LLMs

#90

Post by _venkatasg »

mitthrowaway2 wrote: post #62

LLMs don't only hallucinate because of mistaken statements in their training data. It just comes hand-in-hand with the model's ability to remix, interpolate, and extrapolate answers to other questions that aren't directly answered in the dataset. For example if I ask ChatGPT a legal question, it might cite as precedent a case that doesn't exist at all (but which seems plausible, being interpolated from cases that do…

Most sentences in the world are not about truth or falsity. Training on a high quality corpus isn’t going to fix ‘hallucination’. The complete separation of facts from sentences is what makes LLMs powerful.
Post reply on HN