Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

61–70 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#61

Post by pdonis »

famouswaffles wrote: post #20
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

Language models (the computation) "care" about making "correct" predictions (correct of course being determined by the data). This is why the simplest solution to reducing hallucinations is just making them more competent/better predictors (more neurons, more data). If the prediction can benefit from all the knowledge/reasoning it has accumulated from training, it'll tap that first. One of the biggest tells of halluc…

> Language models (the computation) "care" about making "correct" predictions

Predictions of what? LLMs aren't predicting anything. They are, as the GP says, generating text that appears to be fluent and coherent (as long as we don't inquire into actual semantic relationships with anything else) in the context of the prompt.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#62

Post by d0mine »

einpoklum wrote: post #24

Key sentence for me: > It might be better to say that everything GPT does is a hallucination , since a state of non-hallucination, of checking the validity of something against some external perception, is absent from these models. I try to explain this to people who are obsessed with using ChatGPT to tell them things. So far I've been telling them something like: "it does not attempt to provide you valid information…

The same can be said about our brains too. What we think we see is quite different from what our eyes register (e.g., blind spot, hollow-face, colors on contrast backgrounds, etcchttps://m.youtube.com/watch?v=mf5otGNbkuc

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#63

Post by GuB-42 »

AI hallucinations are the opposite of creativity. They give the most obvious wrong answer possible. For example, ask an AI about an unknown basketball player, it will probably describe it at fit and tall, because that's how we expect basketball players to be. That may be a good observation, but certainly not creativity.

If instead it told the story of someone short and obese, with details on how he got to play basketball, that would be creative. Of course, you can ask an AI to tell the story of a short and obese basketball player, and it will tell you a likely story for such a player, but again that's not creativity, it is just the AI filling the blanks based on whatever similar stories it had in its training set.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#64

Post by pdonis »

jshen wrote: post #44
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

> humans aren't "optimized" to be correct either

Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating.

LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#65

Post by pdonis »

teaearlgraycold wrote: post #16
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> LLMs have no understanding of the underlying reality Not true. They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. The only reason LLMs are getting better is because they are better modeling the world. At some point the only way to improve token prediction is to gain an understanding of the world.

> They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages.

Perhaps the people building LLMs are doing this, but the LLMs themselves are not. LLMs are just generating text. They aren't "reverse engineering" anything.

> The only reason LLMs are getting better is because they are better modeling the world.

No, they are getting better at generating text that seems fluent and coherent, as long as you don't inquire into any actual semantic relationships with the world. LLMs can't model the world because they don't even have a concept of "the world". All they have is text.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#66

Post by Retric »

simplify wrote: post #52
Retric wrote: post #39

Earlier quoted context omitted.

Science progresses because people are testing plausible ideas not random ideas. We don’t do human trials on random drugs, we do human trials on promising ones. Animal trials are more open but even then candidates are carefully considered as being viable. Things are even more open the earlier you are in drug discovery, but we aren’t testing molecules using Dubnium or most elements on the periodic table. Similarly Ecol…

No, science is indeed testing random ideas. Limiting what is tested to an arbitrary plausibility score is something funders and sponsors do, not scientists. What you're talking about is ethics, which is important too.

Random ideas include things like:

The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres.

I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#67

Post by pixl97 »

km3r wrote: post #55
mjr00 wrote: post #34

Earlier quoted context omitted.

> This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art. Yeah but who wants to consume art purely generated by AI (that is, not human-created with AI support)? Most art sites have had blanket bans, or at least required tagging, on ai-generated art because people hate it so much. Or to put it another way: why are you in the comment s…

If they hate it so much it shouldn't need tagging as it would be naturally lower ranked. Different forms of art (ai or human made) are good for different situations. Commenting on hacker news has a different set of needs and depth that ChatGPT cannot replicate. But I do switch over to chatGPT when trying to get background information about things. Different 'tools' for different needs.

>If they hate it so much it shouldn't need tagging as it would be naturally lower ranked.

Eh, this is a two sided argument.

For example, lots of people look at porn. This doesn't mean when I pull up an art/picture site I want to have porn displayed to me. In addition sites that allow porn are typically flooded with massive amounts of that kind of content. If you are a site that wants to show non-pornographic artistic content, in general you have to ban and heavily moderate it or it takes over the site.

I can see where the same will be true for AI generated art. Where for human art, one work could take hours, maybe days or far longer to create. Nearly unlimited amounts of AI art could flood a site in days, so much so that even attempting to host content would not be feasible in any way.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#68

Post by danmaz74 »

jshen wrote: post #44
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of the real world, while LLM can, at most, develop an understanding of human language and the constructs that human language can create. But without the napping that we have between that language and our experience of the world.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#69

Post by pixl97 »

meowtimemania wrote: post #56
wildrhythms wrote: post #41

Earlier quoted context omitted.

Isn't the whole point of these LLMs be that they are non-deterministic? And with OpenAI opaquely 'tweaking' the model, it's no surprise you don't get the same output.

I think they are seeded. With the same seed, and input, the same model will output the same answer.

I don't think that is exactly true... I've seen arguments based on floating point calculations that accruing errors in FP can generate different results.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#70

Post by duskwuff »

swayvil wrote: post #21

So call it fiction. We like fiction. Truth is hard. Maybe too hard for a mere machine. But dramatic narrative and quirky dialogue might be quite doable. 3000 chapter litrpg fantasy generated overnight.

The problem with using LLMs for fiction is that they lack long-term internal consistency and direction. They can sometimes generate a decent vignette, but establishing (and sticking with!) settings, characters, and plot is still beyond them.
Post reply on HN