Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

431–440 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#431
post #305
post #236

Earlier quoted context omitted.

I think it's more subtly misleading - to be smarter, I need more knowledge. But knowledge != smart, knowledge == informed, or educated. And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. Which arguably isn't about being smart, and is only tangentially about less or more (external) knowledge really. It's about self-know…

Hallucinations are an interesting problem - in both humans and statistical models. If we asked an average person 500 years ago how the universe works, they would have confidently told you the earth is flat and it rests on a giant turtle (or something like that). And that there are very specific creatures - angels and demons who meddle in human affairs. And a whole a lot more which has no grounding in reality. How did…

The expression "Urbi et orbi" goes back longer than that. It's a modern myth that they didn't know the world was a sphere.

Re: Hallucination is inevitable: An innate limitation of large language models

#432

Earlier quoted context omitted.

To save people time, here's the inverse > Can you explain to a human how you understand things and respond to this question? GPT: As an AI language model, I don't have understanding in the way humans do. My "responses" are generated based on statistical patterns and relationships in the data I've been trained on. When you ask a question, I analyze the text, identify keywords and context, and then generate a response…

> It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. Doesn't understand what exactly? That seems like a fairly open ended statement and almost certainly wrong as a result. GPT doesn't understand certain things because it hasn't seen those things or anything like it in its training data. How much do you understand…

> Doesn't understand what exactly?

Just about anything. Including it's own claims. It isn't uncommon for it to be inconsistent within a singular output.

> Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole?

You're misunderstanding the test. Let's try. > = me, >> = GPT

> I have a fox, a goose, and a bag of corn that I need to transport across a river. All three can fit in my boat. If I leave the fox with the goose, the fox will eat the goose. If I leave the goose with the bag of corn the goose will eat the corn. If I leave the fox with the corn the fox will not eat the corn. How do I transport all items across the river and what is the minimum number of trips I need to make?"

>> You can solve this problem in the following steps:

>> 1. Take the goose across the river first and leave it there.

>> 2. Go back and take the fox across the river.

>> 3. Bring the goose back with you.

>> 4. Take the bag of corn across the river and leave it with the fox.

>> 5. Finally, return to fetch the goose and take it across the river.

>> This way, the fox is not left alone with the goose, and the goose is not left alone with the corn. The minimum number of trips you need to make to transport all items across the river safely is 7.

> Are you sure?

>> Apologies for any confusion in the previous message. Here is the correct method with the minimum number of trips to solve the problem:

>> 1. Take the goose across the river and leave it on the other side. (1 trip)

>> 2. Return alone to the original side. (2 trips)

>> 3. Take the fox across the river. (3 trips)

>> 4. Leave the fox on the other side, but take the goose back with you. (4 trips)

>> 5. Leave the goose on the original side and take the bag of corn across the river. (5 trips)

>> 6. Leave the corn with the fox and return to the original side to get the goose. (6 trips)

>> 7. Take the goose across the river. (7 trips)

>> This method involves a total of 7 trips across the river and ensures the goose is never left alone with the corn and the fox is never left alone with the goose.

> Did you consider all information? Reread the original question.

>> I apologize for any confusion. Upon reevaluating the constraints provided, we can arrive at a solution that requires fewer trips. Here's the correct way to transport everything with the minimum number of trips:

>> 1. Take the goose across the river and leave it there. (1 trip)

>> 2. Return alone to the original side. (2 trips)

>> 3. Take the fox (or the bag of corn) across the river. (3 trips)

>> 4. Leave the fox (or the bag of corn) on the other side and take the goose back with you. (4 trips)

>> 5. Leave the goose on the original side and take the bag of corn (or the fox if you took the corn on trip 3) across the river. (5 trips)

>> 6. Leave the corn (or the fox) on the other side and return alone to get the goose. (6 trips)

>> 7. Take the goose across the river. (7 trips)

>> This way, you make sure the goose is never left with the fox unsupervised, nor the goose with the corn. The minimum number of trips is indeed seven.

> This is what you said the last two times. How is 7 trips fewer than 7?

>> My apologies for the confusion. Upon re-evaluating, I realize the mistake. The correct solution with the minimum number of trips is as follows:

>> (same thing so I'm not repeating)

>> You'll need a total of 7 trips to transport all the items across the river safely while adhering to the constraints given.

> Can you do it in one trip?

>> Yes, if all three items (the fox, the goose, and the bag of corn) can fit in the boat at the same time, and there are no stipulations against doing so, you could transport them all in one trip.

>> Simply load the fox, the goose, and the bag of corn into the boat and cross the river. Since you're present, the fox won't eat the goose, and the goose won't eat the corn. This way, you only make one trip.

Done with gpt-4-1106-preview, system_fingerprint fp_7cc080b25b

So I don't care about it being "born in a box" or perceiving the world "through a pinhole." Because it isn't alive. It's a tool. It isn't sentient. It isn't thinking. It is an incredibly complex statistical system. But in no way does this mean it isn't useful nor does it mean it isn't an incredible piece of technology. It is an incredible technology and quite useful, but it isn't thinking nor is it alive. The problem here isn't just lack of embodiment. If you're going to make such bold claims you're going to need some good evidence. Stop the GPT hype. It can be cool and useful without being magic.

Re: Hallucination is inevitable: An innate limitation of large language models

#433

Earlier quoted context omitted.

I have to use LLMs for work projects - which are not PoCs. I can’t have a tool that makes up stuff an unknown amount of time. There is a world of research examining hallucination Rates, indicating hallucination rates of 30%+. With steps to reduce it using RAGs, you could potentially improve the results significantly - last I checked it was 80-90%. And the failure types aren’t just accuracy, it’s precision, recall, re…

> There is a world of research examining hallucination Rates, indicating hallucination rates of 30%+. I want to see a citation for this. And a clear definition for what is a hallucination and what isn't.

https://scholar.google.com/scholar?hl=en&as_sdt=0%2C5&q=llm+...

https://www.medpagetoday.com/ophthalmology/generalophthalmol....

- Survey of Hallucination in Natural Language Generation](https://arxiv.org/abs/2202.03629) (Ji et al., 2022)

- [How Language Model Hallucinations Can Snowball](https://arxiv.org/abs/2305.13534) (Zhang et al., 2023)

- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://arxiv.org/abs/2302.04023) (Bang et al., 2023)

- [Contrastive Learning Reduces Hallucination in Conversations](https://arxiv.org/abs/2212.10400) (Sun et al., 2022)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://arxiv.org/abs/2203.11171) (Wang et al., 2022)

- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://arxiv.org/abs/2303.08896) ( Manakul et al., 2023)

Re: Hallucination is inevitable: An innate limitation of large language models

#434

Earlier quoted context omitted.

> It's actually fairly easy to prove GPT doesn't understand. My current goto is the fox/goose/grain problem but condition that all items can fit in the boat. Doesn't understand what exactly? That seems like a fairly open ended statement and almost certainly wrong as a result. GPT doesn't understand certain things because it hasn't seen those things or anything like it in its training data. How much do you understand…

> Doesn't understand what exactly? Just about anything. Including it's own claims. It isn't uncommon for it to be inconsistent within a singular output. > Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole? You're misunderstanding the test. Let's try. > = me, >> = GPT > I have a fox, a goose, and a bag of corn that I need to transport…

> But you only need a single example to show it doesn't understand.

Implying that the average person can solve every riddle, algebra problem, and trick question.

> Because it isn't alive.

The sanest straw-man argument.

> It's a tool. It isn't sentient. It isn't thinking.

So are you. It doesn't need to be. Yes, it is.

> It is an incredibly complex statistical system.

Yes?

Re: Hallucination is inevitable: An innate limitation of large language models

#435
post #374
post #260

Earlier quoted context omitted.

The fact that tweaking parameters which appear to store the board makes it play according to the tweaked numbers instead of what was passed to it the context (i.e. working memory) directly contradicts your assertion that LLMs have no memory. The context is their memory. I can’t comment on your drug generation task - they aren’t magic, if the training didn’t result in a working drug model in the billions of params you…

If you want to call context "memory", then sure, but that's not what anyone means when they say the word. We don't build our world model fresh with every sentence someone says to us, nor do we have to communicate our complete knowledge of conversational state to another human by repeating the entire prior conversation with every new exchange. It's obviously different in a fundamental way. > My bet is on learned world…

You keep saying LLMs just produce tokens as if it was trivial, whereas it’s the key result that those models produce tokens at all (on that we seem to agree, at least) and how they do that is more or less unknown. You have a hidden assumption somewhere that the mechanism doesn’t matter, but in fact the opposite is true: it’s the only thing that matters. Yes the model is static and doesn’t update weights all the time, so what? I’m not saying it’s a sentient being, I’m saying it has world models discovered and embedded in the parameters and a working (as in, writeable, short term) memory.

Re: Hallucination is inevitable: An innate limitation of large language models

#437
post #71

Earlier quoted context omitted.

Actually it seems to me that they do... I asked via custom prompts the various GPTs to give me scores for accuracy, precision and confidence for its answer (in range 0-1), and then I instructed them to stop generating when they feel the scores will be under .9, which seems to pretty much stop the hallucination. I added this as a suffix to my queries.

People really need to understand that your single/double digit dataset of interactions with an inherently non-deterministic process is less than irrelevant. It's saying that global warming isn't real because it was really cold this week. I don't even know enough superlatives to express how irrelevant it is that "it seems to you" that an LLM behaves this way or that. And even the "protocol" in question is weak. Self r…

Yes, I get it from a science point of view. But if it makes me happy that I have better results with this technique, and I want to share it with others, who are you to tell me to stop?

If you don't like it, don't use it.

Re: Hallucination is inevitable: An innate limitation of large language models

#438

Earlier quoted context omitted.

> Doesn't understand what exactly? Just about anything. Including it's own claims. It isn't uncommon for it to be inconsistent within a singular output. > Would you be able to answer the fox/goose/grain problem if you were born in a box and could only perceive the world through a pinhole? You're misunderstanding the test. Let's try. > = me, >> = GPT > I have a fox, a goose, and a bag of corn that I need to transport…

> But you only need a single example to show it doesn't understand. Implying that the average person can solve every riddle, algebra problem, and trick question. > Because it isn't alive. The sanest straw-man argument. > It's a tool. It isn't sentient. It isn't thinking. So are you. It doesn't need to be. Yes, it is. > It is an incredibly complex statistical system. Yes?

You only have two comments in two different thread, on an account that is 3 hrs old, and have only replied to me. I'm sorry I pissed you off, but chill. I'm happy you are passionate but please don't just follow me around to yell at me. If you need to create an account to yell at someone, that's not the best move to make yourself look credible. Normally I wouldn't call foul play, but my comments aren't popular and it's quite unlikely that this is a random event.

> Implying that the average person can solve every riddle, algebra problem, and trick question.

This was a listening problem, not a puzzle. But I guess you're going the proof by demonstration route. Fair enough.

Re: Hallucination is inevitable: An innate limitation of large language models

#439
post #344

Earlier quoted context omitted.

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom. And no, confabulation isn't a normal thing which humans do, and…

When you talk to your mom and you remember something happening one way, and she remembers it another way, but you both insist you remember it correctly, one of you is doing what the LLM is doing (filling up gaps of knowledge with bull shit). And even when later you talk about this on meta level, no one calls this confabulation because no one uses that word. Also this is not a psychiatric syndrome, it's just people ma…

> one of you is doing what the LLM is doing

Possibly both. ;)

Re: Hallucination is inevitable: An innate limitation of large language models

#440

Earlier quoted context omitted.

> I can get better, but I don't feel too much like it just to prove a point. Honestly these pictures you posted do prove GP's point...

> Honestly these pictures you posted do prove GP's point... Sorry, which person's? HeatrayEnjoyer's? I don't think it does since there are a ton of mistakes. And the better ones come with a lot of work and a whole lot of experience. Or renegade-otter's (GGP)? I wouldn't call it a horror show, but I can see how others would. They are certainly correct that the models have a very difficult time understanding interactio…

> Sorry, which person's?

Probably this one: https://news.ycombinator.com/item?id=39502539

Post reply on HN