Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

71–80 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#71

Post by jacurtis »

Honestly AI's hallucinating isn't a whole lot different than real life. Only a small minority of humans fact check anything anymore before believing it.

So "irl", we see people like Alex Jones that get up on their big platforms and start spewing nonsense, but if they sound confident enough and it confirms what you want out of the world, then people latch onto it as fact and don't bother to verify. You see this across the internet. Just today I saw a story on instagram that had been re-posted and the person that was talking about it was many degrees removed from the original, but believed it to be real. Looking through the comments, I had to scroll past 50+ comments to find someone who finally called it out as fake. Everyone else was just posting "no way", "wow, I never knew". You never knew because its completely made up. But when we hear someone speak with confidence and we don't care enough to fact-check, then people just believe it.

This is no different than AI. AI models sound confident and reliable. We assume they are making proclamations based on fact but they aren't always (or "usually" in my experience). Many people blindly believe the AI models because they sound reliable and confident in the way they speak. They never say "i don't know".

What AI is doing is problematic for sure. On one hand I want to take the pitchforks and revolt. But on the other hand I look around and realize, that even if we fixed it or vanquished this enemy, we still have a bunch of talking heads doing the same thing.

Maybe AI hallucinations are actually the most human element of AI.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#72

Post by gojomo »

Retric wrote: post #50
gojomo wrote: post #45

Earlier quoted context omitted.

Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.

What seems plausible depends on your level of suspect matter expertise. Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?

That's not my experience with the leading models, & I'm seeing others observe even when LLMs are wrong, they often supply interesting potential avenues to consider (especially in the coding-assistant/debugging domains).

Also, techniques for tamping-down hallucinations are improving rapidly, with teams finding...

• ways to detect likely hallucinations from patterns in internal activations

• extra conditioning to reduce hallucinations

• success using explicit requests that the LLM lies to train effective classifiers for detecting other unrequested falsehoods

• ways to check output against various ground-truths to detect & correct errors

To focus on the real-but-shrinking number of failure modes will miss the almost unbounded upside from continuous improvements.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#73

Post by colanderman »

GuB-42 wrote: post #63

AI hallucinations are the opposite of creativity. They give the most obvious wrong answer possible. For example, ask an AI about an unknown basketball player, it will probably describe it at fit and tall, because that's how we expect basketball players to be. That may be a good observation, but certainly not creativity. If instead it told the story of someone short and obese, with details on how he got to play basket…

This is what the temperature parameter controls. Increase it for less probable (arguably more creative) output.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#74

Post by pixl97 »

Retric wrote: post #50
gojomo wrote: post #45

Earlier quoted context omitted.

Yes - but even bulk LLM 'hallucinations' are usually plausible, and far from 'random'.

What seems plausible depends on your level of suspect matter expertise. Hallucinations are in general ridiculously incorrect and nowhere close to anything worth testing. Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less? Do you really expect hallucinations to generally pick both plausible and untested targets here?

> Put another way what percentage of molecules are worth testing as a viable treatment for epilepsy? 1 in 100 trillion, less?

Depends at the cost and rate you can test them.

When simulations are fast and cheap you have far more latitude in filtering out 'ridiculously incorrect'.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#75

Post by mrbungie »

0xfae wrote: post #18
ethanbond wrote: post #4

Earlier quoted context omitted.

I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...

Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable. That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature. But then again, many discoveries have been made b…

Just a thought.

Hypothesis making may be interpreted as interpolation/extrapolation in a hypothesis space plus some heuristics to reduce that search space based on previous knowledge/valid hypothesis, how much weight you give to said knowledge and evidence, and some soft and hard logic rules. That is in part what allows (some, not counting Dunning–Kruger here) humans how certain to be about what they're arguing/talking about.

Maybe if the LLM is refeed with how likely (i.e. how many samples/tokens support it's response) is the output in its datasets, it may reevaluate its confidence and rephrase its answer.

In the end, the real problem of hallucinations in LLMs is about its confidence in the correctness/plausability of its own output. But that is something 1) humans can also be guilty of; and 2) that is no purely negative, as it can be exploited to generate new knowledge when applying robust hypothesis validation and testing to said ideas.

As you say in your last paragraph, people who've made discoveries in some areas have been treated as insane when tackling problems from a new perspective or when disregarding previous knowledge. If they weren't so strongheaded about their ideas, maybe we wouldn't even be posting in this forum right now.

PS: Still, I agree LLMs commit laughable mistakes sometimes ;)

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#76

Post by ThrustVectoring »

colanderman wrote: post #73
GuB-42 wrote: post #63

AI hallucinations are the opposite of creativity. They give the most obvious wrong answer possible. For example, ask an AI about an unknown basketball player, it will probably describe it at fit and tall, because that's how we expect basketball players to be. That may be a good observation, but certainly not creativity. If instead it told the story of someone short and obese, with details on how he got to play basket…

This is what the temperature parameter controls. Increase it for less probable (arguably more creative) output.

Randomness alone isn't creative. Creativity is the marriage of the improbable and the valuable. Without novel insight into the valuable outer reaches of concept-space, you're not going to generate creative outputs.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#77

Post by mjr00 »

km3r wrote: post #55
mjr00 wrote: post #34

Earlier quoted context omitted.

> This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art. Yeah but who wants to consume art purely generated by AI (that is, not human-created with AI support)? Most art sites have had blanket bans, or at least required tagging, on ai-generated art because people hate it so much. Or to put it another way: why are you in the comment s…

If they hate it so much it shouldn't need tagging as it would be naturally lower ranked. Different forms of art (ai or human made) are good for different situations. Commenting on hacker news has a different set of needs and depth that ChatGPT cannot replicate. But I do switch over to chatGPT when trying to get background information about things. Different 'tools' for different needs.

> If they hate it so much it shouldn't need tagging as it would be naturally lower ranked.

Lower ranked how? Not everything has a Reddit-esque upvote/downvote system. Plenty of sites let you browse by fandom/character/artist tags and just return chronological results (thinking primarily pixiv + all the booru-likes). Those sites were the ones where there was universal backlash to AI art and all of them require an AI-generated tag now.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#79

Post by pixl97 »

d0mine wrote: post #62
einpoklum wrote: post #24

Key sentence for me: > It might be better to say that everything GPT does is a hallucination , since a state of non-hallucination, of checking the validity of something against some external perception, is absent from these models. I try to explain this to people who are obsessed with using ChatGPT to tell them things. So far I've been telling them something like: "it does not attempt to provide you valid information…

The same can be said about our brains too. What we think we see is quite different from what our eyes register (e.g., blind spot, hollow-face, colors on contrast backgrounds, etcc https://m.youtube.com/watch?v=mf5otGNbkuc

Also one to add to your watch list.

PBS: Perception Deception

https://www.youtube.com/watch?v=HU6LfXNeQM4

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#80

Post by Avshalom »

98codes wrote: post #49
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

When a human reads a bunch of stuff on the internet and regurgitates it as fact, we don't say they hallucinate, we say that they're wrong. Why we baby AI on this front, I have no idea.

Because if "AI" is constantly "wrong" or "lying" it'd be bad for the businesses trying to sell it. So there's this marketing push to frame that recognition of constant obvious failure as anthropomorphizing the AI.

Hallucinating on the other hand feels way more abstract. We speak of humans hallucinating answers rarely enough that they can use it for AI with a straight face. Heck, when we do talk about humans hallucinating it's often-as-not in the context of mind expanding experiences; maybe AI hallucinations are even a good thing!

Post reply on HN