Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hallucination is inevitable: An innate limitation of large language models

arxiv.org

Post reply on HN
Print view View original

351–360 of 491 posts

Re: Hallucination is inevitable: An innate limitation of large language models

#351

Post by godelski »

HeatrayEnjoyer wrote: post #299
renegade-otter wrote: post #254

Earlier quoted context omitted.

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw i…

Why do people say stuff like this that is so demonstrably untrue? SD and GPT4 do not exhibit the behavior described above and they're not even new.

Neither of these comments are accurate. (edit: but renegade-otter is more correct)

Here's 1.5 EMA https://imgur.com/mJPKuIb

Here's 2.0 EMA https://imgur.com/KrPVUGy

No negatives, no nothing just the prompt. 20 steps of DPM++ 2M Karras, CFG of 7, seed is 1.

Can we make it better? Yeah sure, here's some examples: https://imgur.com/Dmx78xV, https://imgur.com/HBTitWm

But I changed the prompt and switched to DPM++ 3M SDE Karras

Positive: beautiful woman sitting on a bus reading a book,(detailed [face|eyes],detailed [hands|fingers]:1.2),Tokyo city,sitting next to a window with the city outside,detailed book,(8k HDR RAW Fuji film:0.9),perfect reflections,best quality,(masterpiece:1.2),beautiful

Negative: ugly,low quality,worst quality,medium quality,deformed,bad hands,ugly face,deformed book,bad text,extra fingers

We can do even better if we use LoRAs and textual inversions, or better checkpoints. But there's a lot of work that goes into making really high quality photos with these models.

Edit: here is switching to Cyberrealistic checkpoint: https://imgur.com/gFMkg0J,

And here's adding some LoRAs, TIs, and prompt engineering:

https://imgur.com/VklfVVC (https://imgur.com/ZrAtluS, https://imgur.com/cYQajMN), https://imgur.com/ci2JTJl (https://imgur.com/9tEhzHF, https://imgur.com/4Ck03P7).

I can get better, but I don't feel too much like it just to prove a point.

Re: Hallucination is inevitable: An innate limitation of large language models

#352

Post by p1esk »

Jensson wrote: post #326
p1esk wrote: post #321

Earlier quoted context omitted.

I think when I write, so the machine must also think when it writes. What is it exactly you do when you “think”? And how is it different from what LLM does? Not saying it’s not different, just asking.

There are probably many, but the most glaring one is that LLMs has to write a word every time it thinks, meaning it can't solve a problem before it starts to write down the solution. That is an undeniable limitation of current architectures, it means that the way the LLM answers your question also matches its thinking process, meaning that you have to trigger a specific style of response if you want it to be smart wi…

Ok, so how do humans solve a problem? Isn’t it also a sequential, step by step process, even if not expressed explicitly in words?

What if instead of words a model would show you images to solve a problem? Would it change anything?

Re: Hallucination is inevitable: An innate limitation of large language models

#353

Post by ottaborra »

p1esk wrote: post #305
lazide wrote: post #236

Earlier quoted context omitted.

I think it's more subtly misleading - to be smarter, I need more knowledge. But knowledge != smart, knowledge == informed, or educated. And the problem is more - how can an LLM tell us it doesn't know something instead of just making up good sounding, but completely delusional answers. Which arguably isn't about being smart, and is only tangentially about less or more (external) knowledge really. It's about self-know…

Hallucinations are an interesting problem - in both humans and statistical models. If we asked an average person 500 years ago how the universe works, they would have confidently told you the earth is flat and it rests on a giant turtle (or something like that). And that there are very specific creatures - angels and demons who meddle in human affairs. And a whole a lot more which has no grounding in reality. How did…

by taking steps to verify everything that was said

Re: Hallucination is inevitable: An innate limitation of large language models

#354

Post by bongodongobob »

skywhopper wrote: post #275
digitalsushi wrote: post #199

Earlier quoted context omitted.

'Adjust accordingly' includes giving up and delivering something similar to what I asked, but not what I asked; is this the point at which the circle is complete and AI has fully replaced my dev team?

One thing a human might do that I’ve never seen an LLM do is ask followup and clarifying questions to determine what is actually being requested.

GPT4 absolutely asks for clarification all the time.

Re: Hallucination is inevitable: An innate limitation of large language models

#355

Post by dkjaudyeqooe »

p1esk wrote: post #352
Jensson wrote: post #326

Earlier quoted context omitted.

There are probably many, but the most glaring one is that LLMs has to write a word every time it thinks, meaning it can't solve a problem before it starts to write down the solution. That is an undeniable limitation of current architectures, it means that the way the LLM answers your question also matches its thinking process, meaning that you have to trigger a specific style of response if you want it to be smart wi…

Ok, so how do humans solve a problem? Isn’t it also a sequential, step by step process, even if not expressed explicitly in words? What if instead of words a model would show you images to solve a problem? Would it change anything?

No, I don't know how other people think but I just focus on something and the answer pops into my head.

I generally only use a step by step process if I'm following steps given to me.

Re: Hallucination is inevitable: An innate limitation of large language models

#356

Post by pixl97 »

samatman wrote: post #325
Fripplebubby wrote: post #177

The core argument in this paper it seems to me from scanning it is that because P != NP therefore LLMs will hallucinate answers to NP-complete problems. I think this is a clever point and an interesting philosophical question (about math, computer science, and language), but I think people are mostly trying to apply this using our commonsense notions of "LLM hallucination" rather than the formal notion they use in th…

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom. And no, confabulation isn't a normal thing which humans do, and…

>confabulation isn't a normal thing which humans do

> A normal person is aware of the limits of their knowledge, for whatever reason, LLMs are not.

Eh, both of these things are far more complicated. People perform minor confabulations all the time. Now, there is a medical term for confabulation to about a more serious medical condition that involves high rates of this occurring coupled with dementia, and would be the less common form. We know with things like eye witness testimony people turn into confabulatory bullshit spewing devices very quickly, though likely due to different mechanisms like recency bias and over writing memories by thinking about them.

Coupled with that, people are very apt to lie about things they do know and can do for a multitude of reasons and attempting to teach an LLM to say "I don't know" when it doesn't know something, versus it just lying to you and saying it doesn't know will be problematic. Just see ChatGPT getting lazy in some of its releases for backfire effects like this.

Re: Hallucination is inevitable: An innate limitation of large language models

#357

Post by nedt »

bonzaidrinkingb wrote: post #333
nedt wrote: post #310

Well humans believe that vacination either kills people or gives them chips for tracking and the top politicians are lizard people drinking the blood of children kept in caves and they had to fake a pandemic to get them out. I'd say an A.I. hallucinating isn't that far off from real humans. It's rather the recipient that needs to interpret any response from either.

Around 12k fatal outcomes have been reported in the EU after vaccination, but it is not certain in all cases that vaccines were the cause. The vaccine tracking chips come from two Microsoft (-affiliate) patents, one about using chips to track body activity to reward in cryptocurrency, and another about putting a vaccine passport chip in the hands of African immigrants. That vaccines contain tracking chips is a fabric…

Here out from the German wikipedia about the lockdown being used to cover up the use of children for their blood: "According to the initial interpretation, the mass quarantine (the "lockdown") does not serve to combat the pandemic, but is intended to provide Trump and his allies with an excuse to free countless children from torture chambers, where adrenochrome is being withdrawn en masse on behalf of the elite." – translated via Google translate, but source is here with Die Zeit as source https://de.wikipedia.org/wiki/QAnon#cite_ref-29

Re: Hallucination is inevitable: An innate limitation of large language models

#358

Post by bonzaidrinkingb »

nedt wrote: post #348
bonzaidrinkingb wrote: post #333

Earlier quoted context omitted.

Around 12k fatal outcomes have been reported in the EU after vaccination, but it is not certain in all cases that vaccines were the cause. The vaccine tracking chips come from two Microsoft (-affiliate) patents, one about using chips to track body activity to reward in cryptocurrency, and another about putting a vaccine passport chip in the hands of African immigrants. That vaccines contain tracking chips is a fabric…

In Germany and Austria we have those Querdenker telegram channels. All examples I‘ve given are coming from there. I‘d really like to say I‘ve made it up. But all you did with my message is also what I‘d do with AI output. It can be trained on wrong data, not understanding the question or make stuff up. Just like a human.

I think you are (subconsciously) strawmanning the anti-vax movements like Querdenker. Most of these believe that mandatory vaccination (or reducing freedom of unvaccinated, or making it economically infeasible/required to work) is bad and goes against individual human rights, and that the risks and benefits of vaccines were not clearly communicated.

So, even if you did not make it up, it is twisting the viewpoints to reduce their legitimacy by tying these to ridiculous theories. One could do similar by cherrypicking vaccine proponents and their ridiculous theories (like claiming COVID came from the wet market).

If these channels are not indexed, I have a hard time believing you, given your misgivings and ridicule on your other statements. If a discussion about "Pandemic was faked to get children out of caves" can be sourced, please do so.

AI output is already more careful and fair and balanced on these matters.

Re: Hallucination is inevitable: An innate limitation of large language models

#359

Post by pixl97 »

flextheruler wrote: post #235
foobarian wrote: post #205

Earlier quoted context omitted.

Well in the example of an NP complete problem, a human might realize they are having trouble coming up with an optimal solution and start analyzing complexity. And once they have a proof might advise you accordingly and perhaps suggest a good enough heuristic.

Is the commenter above you implying humans hallucinate to the level of LLMs? Maybe hungover freshman working on a tight deadline without having read the book do, but not professionals. Even a mediocre employees will often realize they’re stuck, seek assistance, and then learn something from the assistance instead of making stuff up.

People commonly realize when they are stuck... But note, the LLM isn't stuck, it keeps producing (total bullshit) material, and this same problem happens with humans all the time when they go off on the wrong tangent and some supervisory function (such as the manager of a business) has to step in and ask wtf they are up to.

Re: Hallucination is inevitable: An innate limitation of large language models

#360

Post by godelski »

HDThoreaun wrote: post #270
renegade-otter wrote: post #254

Earlier quoted context omitted.

It's statistical prediction. LLMs do not "understand" the world by definition. Ask an image generator to make "an image of a woman sitting on a bus and reading a book". Images will be either a horror show or at best full of weird details that do not match the real world - because it's not how any of this works. It's a glorified auto-complete that only works due to the massive amounts of data it is trained on. Throw i…

I think the situation is a lot more complicated than youre making it out to be. GPT4 for example can be very good at tasks it has not seen in the training data. The philosophy of mind is much more open ended and less understood than you seem to think.

> not seen in the training data

Do you have some evidence for this?

Post reply on HN