Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

111–120 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#111

Honestly AI's hallucinating isn't a whole lot different than real life. Only a small minority of humans fact check anything anymore before believing it. So "irl", we see people like Alex Jones that get up on their big platforms and start spewing nonsense, but if they sound confident enough and it confirms what you want out of the world, then people latch onto it as fact and don't bother to verify. You see this across…

>anything anymore

You can cut that part out of the statement. Humans in general are terrible at fact checking anything outside of walking outdoors and looking up at the sky. Leaving that in points to some glorious past where people were not idiots, that unfortunately never existed.

>But when we hear someone speak with confidence and we don't care enough to fact-check, then people just believe it.

When I was a teenager and was learning how people worked I came to the independent discovery of the Big Lie. Telling small lies with components that the listener may have understood didn't work well. But making up a nearly unbelievable fabrication with something the listener did not understand well at all, and saying it with conviction, works a scary percentage of the time.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#112

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> LLMs have no understanding of the underlying reality that language describes.

LLMs absolutely have concepts that extend beyond just words. As early as 2008 (back when there were no large language models, only "regular" language models), we've been able to demonstrate things that seem to me like the model is learning abstract concepts. For a classic example, see Linguistic Regularities in Continuous Space Word Representations [0], a 2013 paper that talks about a model with a latent space where one can take the vector representations of the words "king", "queen", "man", and "woman" and literally perform the arithmetic `king - (man - woman) ≈ queen`. To me, this clearly demonstrates that the model "understands" the concept of gender, represented by a vector in the model's latent space. The set of numbers that you get from the subtraction `man - woman` represent the concept of the difference between the male and female genders, without needing a specific word tied to that representation of the concept.

It's debatable whether or not that counts as "understanding", but that's more of a semantic debate about what it means to understand, not a debate about the model's internal knowledge of the world and ability to do things with that knowledge.

0: https://aclanthology.org/N13-1090.pdf

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#113
post #90

Earlier quoted context omitted.

Right, this is also why we don't see breakthroughs in science or literature from AI.

How do you recognize a breakthrough in literature ?

When it moves people in a meaningful way, when it explores a new style or narrative that has not been traversed before. As of now, AI struggles to create long form cohesive narratives altogether.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#114
post #63

AI hallucinations are the opposite of creativity. They give the most obvious wrong answer possible. For example, ask an AI about an unknown basketball player, it will probably describe it at fit and tall, because that's how we expect basketball players to be. That may be a good observation, but certainly not creativity. If instead it told the story of someone short and obese, with details on how he got to play basket…

Is creativity something innate in humans, or can it be learned? I would argue that if a human can learn creativity then an AI can too. By the way, here's GPT-4 "Write a description of an unlikely basketball player. Be creative." > Meet Ethel, a 4'9" grandmother of six, with a penchant for knitting and a mastery of Sudoku. With bifocal glasses perched on her nose, she's far from your typical basketball star. But what…

[deleted]

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#115

it's just a consequence of the annoying anthropomorphizing of tech. "Human's can't do X, the AI model can't do X, look it's just like me fr, fr". Of course nobody applies that logic to a forklift or a debugger. If gdb started to hallucinate variables into existence we don't call it a creative act, we call it a bug. Given that these AI systems just like any other machine operate at scale, automated, and fast, they mus…

I don't really think you understand the premise of generalization.

A forklift is not a generalized machine. It is a specific machine for a very well defined set of tasks.

A LLM can take a set of input information, choose a number of options, like using an external tool, and act upon that data.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#116
post #63

AI hallucinations are the opposite of creativity. They give the most obvious wrong answer possible. For example, ask an AI about an unknown basketball player, it will probably describe it at fit and tall, because that's how we expect basketball players to be. That may be a good observation, but certainly not creativity. If instead it told the story of someone short and obese, with details on how he got to play basket…

Is creativity something innate in humans, or can it be learned? I would argue that if a human can learn creativity then an AI can too. By the way, here's GPT-4 "Write a description of an unlikely basketball player. Be creative." > Meet Ethel, a 4'9" grandmother of six, with a penchant for knitting and a mastery of Sudoku. With bifocal glasses perched on her nose, she's far from your typical basketball star. But what…

I've seen some incredibly creative work from Midjourney, even for some of my shitty promots.

I strongly disagree with the basic assumptions of the article. Creativity and imagination are a process of being wrong: imagining that Barney the Dinosaur has a magic box is categorically false, yet it remains creative. In addition, all revolutionary ideas start as falsehoods. Einstein was "wrong" according to what humanity considered "right" when he came up with SR - it eventually became right.

Finally, the fear of being wrong - which is a learned fear - causes real harm. Being wrong (and eventually right) should be celebrated - and we have seen that ChatGPT is usually more than happy to be told that it's wrong.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#117

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> LLMs have no understanding of the underlying reality that language describes. LLMs absolutely have concepts that extend beyond just words. As early as 2008 (back when there were no large language models, only "regular" language models), we've been able to demonstrate things that seem to me like the model is learning abstract concepts. For a classic example, see Linguistic Regularities in Continuous Space Word Repre…

Are you sure the model in the paper linked and LLMs function similarly or identically in this regard?

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#118
post #95

Earlier quoted context omitted.

> They clearly do as plenty research indicates. I have seen plenty of researchers claim this. What I have not seen is actual support for such a claim.

There's plenty support. You just need to know how to read.

Please link to published papers that prove what you’re positing.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#119

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> LLMs have no understanding of the underlying reality that language describes. LLMs absolutely have concepts that extend beyond just words. As early as 2008 (back when there were no large language models, only "regular" language models), we've been able to demonstrate things that seem to me like the model is learning abstract concepts. For a classic example, see Linguistic Regularities in Continuous Space Word Repre…

I would suggest that example is more likely evidence of semantic associations provided by underlying taxonomy based on input training data. There is so much contextual information in language hierarchy and associated characteristics and there is nothing stopping a LLM from using multiple recombinant factors for generation.

I don’t mean to discount your excellent comment. I mean obviously these models are so complex already that their black box nature makes it difficult to derive conclusive results, but Arkham’s razor would imply that the simplest explanation — that is it recombined training data creating a heirarchy with multiple factors that provides context for the “correct” answer — that is more likely correct.

In other words a LlM can confer multiple related associations to output based on training data, and that is the most likely explanation for that behavior.

Just look at the poor performance with GPT-3 and the underlying data quality issues the Alpaca dataset team has discussed.

I’m no expert, and don’t mean to counter you as much as contribute my own limited thinking for discussion here.

Cheers!

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#120
post #34

This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art. The article says "well, sometimes what it makes is bad". Well big deal. A lot of human-created art is awful too.

> This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art. Yeah but who wants to consume art purely generated by AI (that is, not human-created with AI support)? Most art sites have had blanket bans, or at least required tagging, on ai-generated art because people hate it so much. Or to put it another way: why are you in the comment s…

> Or to put it another way: why are you in the comment section of Hacker News, and not just asking ChatGPT to generate social media comments on the article?

Current models have fairly poor performance, compared to a human. If better art or better conversation could come from AI, then I suspect many might prefer it. Why wouldn't we? Why would we want simpler, less enjoyable, "real" conversations, or less symbolic art?

Post reply on HN