Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

101–110 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#101

Post by PoignardAzur »

GuB-42 wrote: post #63

AI hallucinations are the opposite of creativity. They give the most obvious wrong answer possible. For example, ask an AI about an unknown basketball player, it will probably describe it at fit and tall, because that's how we expect basketball players to be. That may be a good observation, but certainly not creativity. If instead it told the story of someone short and obese, with details on how he got to play basket…

I don't know, asking ChatGPT4 to be creative seems to work okay:

https://chat.openai.com/share/62180301-b7ae-46fe-bd15-bd6973...

> Jack "Shadow" Carter was a prodigy dismissed for his short stature, standing only 5'7". Ignored by scouts and overshadowed by taller players in high school, he developed a unique playing style that exploited his low center of gravity and agility. He became a master of steals and assists, zipping around the court like a shadow, hence his nickname.

(Though I get your point that "hallucinations" will tend towards lowest-common-denominator answers, not creative answers.)

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#102

Post by rollcat »

Hallucination... that's an anthropomorphism. LLMs are functions. They don't make things up, they just sometimes return non-factual data. Which makes complete sense; the model does not have the notion of a fact, otherwise we would be talking about something more like expert systems, or knowledge graphs, not LLMs.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#104

Post by swayvil »

duskwuff wrote: post #70
swayvil wrote: post #21

So call it fiction. We like fiction. Truth is hard. Maybe too hard for a mere machine. But dramatic narrative and quirky dialogue might be quite doable. 3000 chapter litrpg fantasy generated overnight.

The problem with using LLMs for fiction is that they lack long-term internal consistency and direction. They can sometimes generate a decent vignette, but establishing (and sticking with!) settings, characters, and plot is still beyond them.

I imagine a componentized approach to fiction generation. The list of characters is one component. The narrative arc (in the abstract) is another. The world is another. Each dialog chunk, each action chunk, another.

Each component AI generated. And each referring to the other components for its construction. For consistency.

It could be horribly formulaic but cunning and tasty too.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#105

Post by famouswaffles »

SirMaster wrote: post #97
famouswaffles wrote: post #54

Earlier quoted context omitted.

>The AI doesn't "know" what it "knows". Yes it does. https://news.ycombinator.com/item?id=37874556

OK, how about we put it this way? Who cares if "it knows" if it's apparently impossible to get it to use that knowledge to stop hallucinating? To me, the end user there is no practical difference between not having the data and not being able to use the data it has. If it can't use it, or refuses to use it, it may as well not exist.

If you'd bothered to look at the linked papers, you'd see it's not "impossible" to get much better calibration. Just difficult and something that could be further worked on.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#106

Post by famouswaffles »

pdonis wrote: post #95
famouswaffles wrote: post #87

Earlier quoted context omitted.

>LLMs don't even have a concept of "being correct" They clearly do as plenty research indicates. You've just decided not to accept this. Pure confirmation bias in action.

> They clearly do as plenty research indicates. I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.

There's plenty support. You just need to know how to read.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#107

Post by famouswaffles »

pdonis wrote: post #96
famouswaffles wrote: post #86

Earlier quoted context omitted.

They are quite literally trained to predict a distribution that could supply the next token given previous context. This is their primary objective.

> They are quite literally trained to predict a distribution that could supply the next token given previous context. "Previous context" here means text . It does not mean "the actual world". Big difference.

So ? That's not a big difference lol. The world can be sufficiently represented in text. and bolting in additional modalities to a transformer is trivial. Could just as well be images and audio also.

You think you experience the "true" world ? You don't. You experience a slice of it that your brain often further fabricates at parts.

To the birds that feel and sense electromagnetic waves intuitively to guide travels, your model of vision and direction is fundamentally incomplete/incorrect. No one gets to experience the real world.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#108

Post by ta8645 »

SirMaster wrote: post #97
famouswaffles wrote: post #54

Earlier quoted context omitted.

>The AI doesn't "know" what it "knows". Yes it does. https://news.ycombinator.com/item?id=37874556

OK, how about we put it this way? Who cares if "it knows" if it's apparently impossible to get it to use that knowledge to stop hallucinating? To me, the end user there is no practical difference between not having the data and not being able to use the data it has. If it can't use it, or refuses to use it, it may as well not exist.

> Who cares if "it knows" if it's apparently impossible to get it to use that knowledge to stop hallucinating?

It sounds like you're describing the MSM. In any case, the same problem is true for a fair number of people. The difference is, silicon training has a much better chance of evolving beyond its current limitations much sooner than the typical human.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#109

Post by PoignardAzur »

pdonis wrote: post #65
teaearlgraycold wrote: post #16

Earlier quoted context omitted.

> LLMs have no understanding of the underlying reality Not true. They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. The only reason LLMs are getting better is because they are better modeling the world. At some point the only way to improve token prediction is to gain an understanding of the world.

> They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. Perhaps the people building LLMs are doing this, but the LLMs themselves are not. LLMs are just generating text. They aren't "reverse engineering" anything. > The only reason LLMs are getting better is because they are better modeling the world. No, they are getting better at generating text that…

This subjects comes up in every single discussion on LLMs, and every time someone asserts with perfect confidence that LLMs couldn't possibly have a world model.

But that confidence is based on intuition, not on any empirical observations. All the research we've seen so far points in the other direction: sufficiently trained LLMs do have a world model that you can find in their weights, changing these weights changes their completions in a way consistent with the new world model, etc. See OthelloGPT for the most blatant example.

I'm starting to wish we had something like a FAQ to point to, with a summary of all the research on the subject, because it's pretty settled by now.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#110

Post by nuancebydefault »

pdonis wrote: post #64
jshen wrote: post #44

Earlier quoted context omitted.

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

> humans aren't "optimized" to be correct either Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating. LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.

Humans do not _know_ the external world either. All humans have is perceptions from media like sounds, images and text! Internally we try to make a model of the world based on those perceptions. Any blanks are filled in by creativity ! Think about it, when we write or speak we start from some internalized model (which is mostly imperfect) and then we build text around it that feels fluent and consistent.

I would argue, human reasoning is conceptually not so much different from LLMs as one might think.

Post reply on HN