Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

51–60 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#51

Post by km3r »

jshen wrote: post #44
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

There is a difference between general AI and generative AI. Humans can be creative and create vast fictional worlds with made up technology and magic. They can also optimize for facts and pass a BAR exam.

LLM like have some form of understanding of the world, not sure if its anywhere near comparable to our understanding though. But they are still not general enough to really focus on facts. They can get close but the nature of statistics means there isn't hard checks to truth.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#52

Post by simplify »

Retric wrote: post #39
gojomo wrote: post #32

Contra the headling, we should "celebrate being wrong" – when it's wrong in fast, interesting, & voluminous ways that can then be filtered or corrected. That's how lots of science, innovation, & learning work: generate many superficially-plausible candidates via a fast-and-loose process, then refine with a more rigorous evaluation. That AIs, in the form of LLMs, are now doing this so well was unexpected, and progress…

Science progresses because people are testing plausible ideas not random ideas. We don’t do human trials on random drugs, we do human trials on promising ones. Animal trials are more open but even then candidates are carefully considered as being viable. Things are even more open the earlier you are in drug discovery, but we aren’t testing molecules using Dubnium or most elements on the periodic table. Similarly Ecol…

No, science is indeed testing random ideas. Limiting what is tested to an arbitrary plausibility score is something funders and sponsors do, not scientists.

What you're talking about is ethics, which is important too.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#53

Post by daoboy »

The term "hallucinating" in this context is just another way we anthropomorphize machines. It is mostly harmless but I see the author's point that it can be misleading when trying to address specific issues.

On the (rare) occasion I find it useful to avoid this very normal tendency I ask myself if it would make sense to apply the same framing to the output of an AI image generator.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#54

Post by famouswaffles »

farnsworth wrote: post #25
Sohcahtoa82 wrote: post #17

I never liked that the term "hallucinating" ended up sticking. The AI doesn't know something so it just invents something. We usually call that "bullshitting", or in more polite crowds, "lying".

That's not a useful description. The AI doesn't "know" what it "knows". It's not even filling some gap in its knowledge. It's just putting words together that statistically can go together.

>The AI doesn't "know" what it "knows".

Yes it does.

https://news.ycombinator.com/item?id=37874556

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#55

Post by km3r »

mjr00 wrote: post #34
LeoPanthera wrote: post #22

This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art. The article says "well, sometimes what it makes is bad". Well big deal. A lot of human-created art is awful too.

> This is an odd article. To me, it seems like the "creative" arts are an ideal arena for AI. After all, there's no such thing as "wrong" art. Yeah but who wants to consume art purely generated by AI (that is, not human-created with AI support)? Most art sites have had blanket bans, or at least required tagging, on ai-generated art because people hate it so much. Or to put it another way: why are you in the comment s…

If they hate it so much it shouldn't need tagging as it would be naturally lower ranked.

Different forms of art (ai or human made) are good for different situations. Commenting on hacker news has a different set of needs and depth that ChatGPT cannot replicate. But I do switch over to chatGPT when trying to get background information about things. Different 'tools' for different needs.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#56

Post by meowtimemania »

wildrhythms wrote: post #41
fragmede wrote: post #27

Curiously enough, I don't get the same result as the author on plain ChatGPT-4. https://chat.openai.com/share/e213e0bd-2838-45e2-9942-e52954... https://chat.openai.com/share/64bc62d3-042c-40c4-8d50-8e28ce... Hilariously, plugging the example in the article into Bing enhanced ChatGPT-4, ChatGPT-4 w/ Bing hallucinates because of that very article! https://chat.openai.com/share/4fe50933-8436-44ad-a778-6297ca... If you t…

Isn't the whole point of these LLMs be that they are non-deterministic? And with OpenAI opaquely 'tweaking' the model, it's no surprise you don't get the same output.

I think they are seeded. With the same seed, and input, the same model will output the same answer.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#57

Post by pixl97 »

jshen wrote: post #44
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

>I always find this point a bit odd because humans aren't "optimized" to be correct either.

It depends which subsystem.

The older the system in the body the less likely it is to have a high error rate, otherwise we'd die from cancer at a much higher rate or injure ourselves far more often. Of course this also depends on the definition of 'correct', if the system never changed we'd never evolve.

>> "LLMs have no understanding of the underlying reality"

This statement has always bothered me because it really depends on what you mean by 'underlying reality'. How many layers are we talking about? What does understanding mean? Because at the end of the day, humans don't really understand 'underlying reality, we just have a particular set of input devices we take in information and do some transformations on it... Where is the understanding happening?

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#58

Post by pdonis »

happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> All these criticisms of hallucinations are missing the point.

Not if people need to be reminded that, as you say, LLMs are not designed to give reliable answers. Many, many people appear to believe that they are, since they rely on the answers in all kinds of contexts. For example, in past HN threads on LLMs, I have seen people say that they rely on LLM-generated code.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#59

Post by jrockway »

ethanbond wrote: post #4
glasss wrote: post #2

I really didn't think anyone was applauding the "hallucinations" but maybe I just hang around people who aren't visionaries.

I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...

I think people like the hallucinations because they seem like the right answer. Imagine I make an AI assistant. You ask it, "how do I catch the bus into town?" If it replies, "run around in your backyard in a circle, then clap 3 times", (i.e. just use a random number generator) you'll think that's junk and will hate the system. But if it says "go to the corner of Main and Pine at 6:53 AM", then you'll probably do that. No matter that there is no bus route on Main or Pine. It sounds right, so you'll want to follow up. I imagine the same applies to searching for novel molecules. "This looks kind of right, I never would have thought of that", so you get a grant to synthesize it, do some clinical trials, and find that it doesn't work at all. But hey, you had something to do for a couple years, and that outcome happened in the non-AI process all the time too. By the time there is enough data that points strongly at the AI suggestions being as good as random chance, the AI gold rush will be over and cashed out. Or, the hallucinations will turn out to be really good and we'll cure cancer or whatever. What matters is that whoever made the model makes a lot of money either way.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#60

Post by tikhonj »

jshen wrote: post #44
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

Humans have evolved to communicate based on some sort of shared mental model and shared intentionality—human speech generally has some sort of more-or-less coherent semantics by its very nature. While it's hard to reason about what goes on inside the neural network, there are too many examples of LLMs outputting not-even-wrong nonsense to conclude they have a similar internal model, even assuming that having an internal model at all would make sense in this context!
Post reply on HN