Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

11–20 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#11

Post by happytiger »

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time.

1) Hallucinations often appear because LLMs are designed to create fluent, coherent text.

2) LLMs have no understanding of the underlying reality that language describes.

3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt.

It sacrifices accuracy for being good at conversations as it is designed to do. All these criticisms of hallucinations are missing the point.

Generative AI is generative and basically is a specialist at making things up. It’s going to take things like multiprompting, network AIs that fact check output, and a host of other technologies or even entirely new models of AI to solve these problems, but don’t make the mistake of thinking that the system is supposed to be working without hallucinations right now — that’s not what it’s optimized for.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#15

Post by thfuran »

happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

Did you read the article?

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#16

Post by teaearlgraycold »

happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> LLMs have no understanding of the underlying reality

Not true. They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. The only reason LLMs are getting better is because they are better modeling the world. At some point the only way to improve token prediction is to gain an understanding of the world.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#18

Post by 0xfae »

ethanbond wrote: post #4
glasss wrote: post #2

I really didn't think anyone was applauding the "hallucinations" but maybe I just hang around people who aren't visionaries.

I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...

Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable.

That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature.

But then again, many discoveries have been made because someone trying something that shouldn't work.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#19

Post by vinceguidry »

thfuran wrote: post #15
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

Did you read the article?

skims article

Nope, they summarized it without reading it. Useful enough for those who want to keep a finger to the pulse of AI without a huge time investment.

God save me from having to read every article on HN.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#20

Post by famouswaffles »

happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

Language models (the computation) "care" about making "correct" predictions (correct of course being determined by the data). This is why the simplest solution to reducing hallucinations is just making them more competent/better predictors (more neurons, more data). If the prediction can benefit from all the knowledge/reasoning it has accumulated from training, it'll tap that first. One of the biggest tells of hallucinations is if the model is saying something completely different each generation vs being consistent.

What they just don't care about is communicating being out of distribution or making whack predictions. This becomes a problem for humans because they're perfectly fine making things up when the above fail.

But by all accounts, they do learn to distinguish these things. The computation is very much aware when it is going way off base.

GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r

Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975

Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334

Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets - https://arxiv.org/abs/2310.06824

Post reply on HN