Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

381–390 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#381

Post by renegade-otter »

godelski wrote: post #351
HeatrayEnjoyer wrote: post #299

Earlier quoted context omitted.

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

Neither of these comments are accurate. (edit: but renegade-otter is more correct) Here's 1.5 EMA https://imgur.com/mJPKuIb Here's 2.0 EMA https://imgur.com/KrPVUGy No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1. Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV , https://imgur.com/HBTitWm But I changed the prompt and switched to DPM++ 3M SDE…

You kind of proved my point. Of course the "finger situation" is getting better but people handling complex objects is still where these tools trip. They can't reason about it - they just need to see enough data of people handling books. On a bus. Now do this for ALL possible objects in the world.

I have generated hundreds of these - the bus cabin LOOKS like a bus cabin, but it's a plausible fake - the poles abruptly terminate, the seats are in weird unrealistic configurations, unnatural single-row isles, etc. Which is why I called it a super-convincing autocomplete.

Re: Hallucination is inevitable: An innate limitation of large language models

#382

Post by pixl97 »

jncfhnb wrote: post #308
sandworm101 wrote: post #163

Earlier quoted context omitted.

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from curre…

> They cannot say "I dont know" because they dont actually know anything. print(“I don’t know”) You don’t need proper cognition to identify that the answer is not stored in source data. Your conception of the model is incomplete as is easily demonstrable by testing such cases now. Chat gpt does just fine on your simpsons test. You, however, have made up an answer of how something works that you don’t actually know de…

>You don’t need proper cognition to identify that the answer is not stored in source data.

Uh, what?

So lets imagine you have an LLM that knows everything, except you withhold the data that you can put peanut butter on toast. Toast + Peanut butter = does not exist in data set. So what exactly do you expect the LLM to say when someone asks "Can you put peanut butter on toast?".

I would expect an intelligent agent to 'think' Peanut butter = spreadable food, toast = hard food substrate, so yea, they should work instead of the useless answer of I don't know.

Everything that does not exist in nature is made up by humans, the question is not "is it made up" the question is "does it work"

Re: Hallucination is inevitable: An innate limitation of large language models

#383

Post by renegade-otter »

HDThoreaun wrote: post #270
renegade-otter wrote: post #254

Earlier quoted context omitted.

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw i…

I think the situation is a lot more complicated than youre making it out to be. GPT4 for example can be very good at tasks it has not seen in the training data. The philosophy of mind is much more open ended and less understood than you seem to think.

What's the evidence? OpenAI's claims? They do have an inherent interest is making investors believe this technology is magic.

Re: Hallucination is inevitable: An innate limitation of large language models

#384

Post by pixl97 »

AlexandrB wrote: post #288
DiogenesKynikos wrote: post #264

Earlier quoted context omitted.

The claim was made that LLMs just parrot back what they've seen in the training data. They clearly go far beyond this and generate completely novel ideas that are not in the training data. I can give ChatGPT extremely specific and weird prompts that have 0% chance of being in its training data, and it will answer intelligently. > The actual construction of a neural network llm refutes your assertions. I don't see how…

> They clearly go far beyond this and generate completely novel ideas that are not in the training data. There's a case where this is trivially false. Language. LLMs are bound by language that was invented by humans. They are unable to "conceive" of anything that cannot be described by human language as it exists, whereas humans create new words for new ideas all the time.

Uh, I believe you're really confused on things like ChatGPT versus LLMs in general. You don't have to feed human language to an LLM for them to learn things. You can feed wifi data waveforms for example and they can 'learn' insights from that.

Furthermore you're thinking here doesn't even begin to explain multimodal models at all.

Re: Hallucination is inevitable: An innate limitation of large language models

#385

Post by DiogenesKynikos »

Jensson wrote: post #318
DiogenesKynikos wrote: post #263

Earlier quoted context omitted.

Then your definition of understanding is meaningless. If a physical system is able to accurately simulate understanding, it understands.

A human that mimics the speech of someone that does understand usually doesn't understand himself. We see that happen all the time with real humans, you have probably seen that as well. To see if a human understands we ask them edge questions and things they probably haven't seen before, and if they fail there but just manage for common things then we know the human just faked understanding. Every LLM today fails thi…

> A human that mimics the speech of someone that does understand usually doesn't understand himself.

That's not what LLMs do. They provide novel answers to questions they've never seen before, even on topics they've never heard of, that the user just made up.

> To see if a human understands we ask them edge questions

This is testing if there are flaws in their understanding. My dog understands a lot of things about the world, but he sometimes shows that he doesn't understand basic things, in ways that are completely baffling to me. Should I just throw my hands in the air and declare that dogs are incapable of understanding anything?

Re: Hallucination is inevitable: An innate limitation of large language models

#386

Post by DiogenesKynikos »

AlexandrB wrote: post #288
DiogenesKynikos wrote: post #264

Earlier quoted context omitted.

The claim was made that LLMs just parrot back what they've seen in the training data. They clearly go far beyond this and generate completely novel ideas that are not in the training data. I can give ChatGPT extremely specific and weird prompts that have 0% chance of being in its training data, and it will answer intelligently. > The actual construction of a neural network llm refutes your assertions. I don't see how…

> They clearly go far beyond this and generate completely novel ideas that are not in the training data. There's a case where this is trivially false. Language. LLMs are bound by language that was invented by humans. They are unable to "conceive" of anything that cannot be described by human language as it exists, whereas humans create new words for new ideas all the time.

I just asked ChatGPT to make up a Chinese word for hungry+angry. It came up with a completely novel word that actually sounds okay: 饥怒. It then explained to me how it came up with the word.

You can't claim that that isn't understanding. It just strikes me that we've moved the goalposts into every more esoteric corners: sure, ChatGPT seems like it can have a real conversation, but can it do X extremely difficult task that I just thought up?

Re: Hallucination is inevitable: An innate limitation of large language models

#387

Post by paulmd »

dragonwriter wrote: post #249
someplaceguy wrote: post #243

Earlier quoted context omitted.

> their function is to produce text output which forms a plausible seeming response to the question posed Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. And it would not be a hallucination.

> Answering "I don't know" or "I can't answer that" is a perfectly plausible response to a difficult logical problem/question. Sure, and you can train LLMs to produce answers like that more often, but then users will say your model is lazy and doesn't even try, whereas if you train it to be more likely to produce something that looks like a solution more often, people will think “wow, the AI solved this problem I cou…

if more guiderails are useful to users then such things will surely emerge.

but from an engineering perspective it makes sense to have a "generalist model" underneath that is capable of "taking its best guess" if commanded, and then trying to figure out how sure it is about its guess, build guiderails, etc. Rather than building a model that is implicitly wishy-washy and always second-guessing itself etc.

The history of public usage of AI has basically been that too many guiderails make it useless, not just gemini making japanese pharohs to boost diversity or whatever, but frankly even mundane usage is frustratingly punctuated by "sorry I can't tell you about that, I'm just an AI". And frankly it seems best to just give people the model and then if there's domains where a true/false/null/undefined approach makes sense then you build that as a separate layer/guiderail on top of it.

Re: Hallucination is inevitable: An innate limitation of large language models

#388

Post by mannykannot »

cultureswitch wrote: post #332
mannykannot wrote: post #155

Earlier quoted context omitted.

This is a fair question: LLMs do challenge the easy assumption (as made, for example, in Searle's "Chinese Room" thought experiment) that computers cannot possibly understand things. Here, however, I would say that if an LLM can be said to have understanding or knowledge of something, it is of the patterns of token occurrences to be found in the use of language. It is not clear that this also grants the LLM any under…

Should it matter how the object of debate interacts and probes the external world? We sense the world through specialized cells connected to neurons. There's nothing to prevent LLMs doing functionally the same thing. Both human brains and LLMs have information inputs and outputs, there's nothing that can go through one which can't go through the other.

A current LLM does not interact with the external world in a way that would seem to lead to an understanding of it. It emits a response to a prompt, and then reverts to passively waiting for the next one. There's no way for it to anticipate something will happen in response, and thereby get the feedback needed to realize that there is more to the language it receives than is contained in the statistical relationships between its tokens. If its model is updated in the interim, it is unaware, afterwards, that a change has occurred.

Re: Hallucination is inevitable: An innate limitation of large language models

#389

Post by p1esk »

ebcode wrote: post #370
p1esk wrote: post #321

Earlier quoted context omitted.

I think when I write, so the machine must also think when it writes. What is it exactly you do when you “think”? And how is it different from what LLM does? Not saying it’s not different, just asking.

That's a difficult question to answer, since I must be doing a lot of very different things while thinking. For one, I'm not sure I'm never not thinking. Is thinking different from "brain activity"? We can shut down the model, store it on disk, and boot it back up. Shut down my brain and I'm a goner. I'm open to saying that the machine is "thinking", but I do think we need more clear language to distinguish between m…

I don’t think that “think” is a wrong word here. I believe people are machines - more complicated than GPT4, but machines nevertheless. Soon GPT-N will become more complicated than any human, and it will be more capable, so we might start saying that whatever humans do when they think is simpler or otherwise inferior to what the future AI models will do when they “think”.

Re: Hallucination is inevitable: An innate limitation of large language models

#390

Post by smusamashah »

foobiekr wrote: post #224
ryanwaggoner wrote: post #165

Earlier quoted context omitted.

I’m really curious how you managed that. I pasted your exact prompt and GPT-3.5 gave me this: === Making 20-year predictions about the future of Michael Jackson is challenging due to his passing in 2009. However, his legacy as a cultural icon and musical genius will likely endure for decades to come. His music will continue to influence future generations, and his impact on pop culture will remain significant. Additi…

GPT-4’s tendencies to write these long winded but mostly empty responses is so frustrating.

Add "be terse" to your prompts
Post reply on HN