Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

131–140 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#131

Post by nomel »

Barrin92 wrote: post #43

it's just a consequence of the annoying anthropomorphizing of tech. "Human's can't do X, the AI model can't do X, look it's just like me fr, fr". Of course nobody applies that logic to a forklift or a debugger. If gdb started to hallucinate variables into existence we don't call it a creative act, we call it a bug. Given that these AI systems just like any other machine operate at scale, automated, and fast, they mus…

I'm trying to understand this type of perspective, which is somewhat at odds with mine, so the following questions are completely genuine: Which models have you used? What did you try with them? Have you used GPT-4? How frequently?

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#132

Post by happytiger »

km3r wrote: post #51
jshen wrote: post #44

Earlier quoted context omitted.

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

There is a difference between general AI and generative AI. Humans can be creative and create vast fictional worlds with made up technology and magic. They can also optimize for facts and pass a BAR exam. LLM like have some form of understanding of the world, not sure if its anywhere near comparable to our understanding though. But they are still not general enough to really focus on facts. They can get close but the…

I think the next step logically is to see how the human brain functions. We have a similar model where certain parts of our brain are hopelessly useless by themselves as they are so overly specialized in one thing, but work in harmony to create network effectiveness.

And we only just [mapped it for the first time](https://phys.org/news/2023-10-scientists-generate-single-cel...) so now much of what we are learning can potentially be applied to our “digital twins” in coming years.

I imagine a lot of progress with AI will involve similar networks, with governance providing evolutionary paths and guidance for multiple concurrent goals.

There are also lots of issues with training data and memory limits that make LLMs weak at continuity. Holes or weaknesses in the training data might “come through” in the behavior of hallucination.

Where do you think general AI is going right now that we should be looking at? I’m overwhelmed by how large this field has become in the last five years and am always interested in what others know.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#133

Post by happytiger »

Avshalom wrote: post #80
98codes wrote: post #49

Earlier quoted context omitted.

When a human reads a bunch of stuff on the internet and regurgitates it as fact, we don't say they hallucinate, we say that they're wrong. Why we baby AI on this front, I have no idea.

Because if "AI" is constantly "wrong" or "lying" it'd be bad for the businesses trying to sell it. So there's this marketing push to frame that recognition of constant obvious failure as anthropomorphizing the AI. Hallucinating on the other hand feels way more abstract. We speak of humans hallucinating answers rarely enough that they can use it for AI with a straight face. Heck, when we do talk about humans hallucina…

We actually have a core personality characteristic in human psychology for just this thing: neuroticism.

It may be part of complex networks and intelligence, and humans may have a lot of it happening all the time but have ways of compensation or suppression of neurotic or hallucinating thoughts.

We have some intense “mental illnesses” that look a lot like what AI is presenting, where it can’t tell what is going on and fills in the gaps (schizophrenia maybe) or uses behavioral patterns to compensate for memory or intelligence inadequacy.

Obviously straight conjecturing here.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#134

Post by Retric »

simplify wrote: post #92
Retric wrote: post #66

Earlier quoted context omitted.

Random ideas include things like: The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres. I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.

I assure you, there are many scientists that would love to test that, given the resources to do so. Of course no one would fund that, meaning it's the funders deciding on what they think is plausible.

I know plenty of scientists that might approach it as a joke. But they are far too busy to regularly spend significant time on such a things.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#135

Post by IIAOPSW »

happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

In other words, LLM's are, in the technical sense defined by Harry Frankfurt, bullshitters.

The essence of bullshit (as he defines it) is that it is neither truth nor lie but rather unconcerned with veracity altogether. A bullshitter wants to appear smart, or convince others to agree with them, or some other conversational objective. Bullshit statements may or may not ultimately turn out to be true, but the thing that makes them bullshit is that the bullshitter didn't actually know when they said it.

Hence I propose a far more succinct term to describe the phenomena at hand. A far more accurate terminology than the presently vogue "hallucinating": AB. Artificial Bullshit.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#136

Post by Nevermark »

pnpnp wrote: post #100
ryukoposting wrote: post #99

Spot-on article, for the most part. I firmly believe that the use of an LLM as a fact-finding tool represents a misunderstanding of the technology's abilities and intent. If you're using an LLM like it's a search engine, then you should just use a search engine instead. LLMs are conversational devices. Search engines are an index of human knowledge. Right tool, right job. I'm fine with the term "hallucination," thoug…

I’m not sure I agree with this one. My Google results have gotten pretty awful lately. I’ll usually go to an LLM for a very specific search term, then Google to find a reference or validate.

A few months ago, I didn't imagine I would have completely dumped Google for a huge quality improvement in knowledge acquisition and navigation. The Internet is shiny again!! :)

I use GPT4 for questions, and Kagi (paid) for search.

I would love to dump Amazon's search. Maybe someone can scrape Amazon (and other stores) and create a universal agnostic LLM product concierge. Please!

Besides the conflicts of interest ads/trash-products, searching for a product based on highly specific specs has always been hard.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#137

Post by happytiger »

pdonis wrote: post #58
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

> All these criticisms of hallucinations are missing the point. Not if people need to be reminded that, as you say, LLMs are not designed to give reliable answers. Many, many people appear to believe that they are, since they rely on the answers in all kinds of contexts. For example, in past HN threads on LLMs, I have seen people say that they rely on LLM-generated code.

In the land of the blind the one eyed man is king?

;)

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#138

Post by happytiger »

PoignardAzur wrote: post #109
pdonis wrote: post #65

Earlier quoted context omitted.

> They are slowly gaining an understanding of reality by reverse engineering the relationships built into human languages. Perhaps the people building LLMs are doing this, but the LLMs themselves are not. LLMs are just generating text. They aren't "reverse engineering" anything. > The only reason LLMs are getting better is because they are better modeling the world. No, they are getting better at generating text that…

This subjects comes up in every single discussion on LLMs, and every time someone asserts with perfect confidence that LLMs couldn't possibly have a world model. But that confidence is based on intuition, not on any empirical observations. All the research we've seen so far points in the other direction: sufficiently trained LLMs do have a world model that you can find in their weights, changing these weights changes…

Any network of sufficiently complex connected nodes develops characteristics of complexity that mirror intelligence.

This was always the conjecture of the “ghost in the machine” and mech futurist sci-fi writers.

Would you be willing to provide some reading about this? I’d love to see 1-2 things that could help me (and others) know more about what you know.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#139

Post by happytiger »

thfuran wrote: post #15
happytiger wrote: post #11

The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time. 1) Hallucinations often appear because LLMs are designed to create fluent, coherent text. 2) LLMs have no understanding of the underlying reality that language describes. 3) LLMs use statistics to generate language that is grammatically and semantically correct within the context of the prompt. It…

Did you read the article?

I did. I’m aware. But I thought the comment worth writing anyways.

I’m sorry it wasn’t enough of a response to the article for you, but that wasn’t my intention in writing it.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#140

Post by galaxyLogic »

Nevermark wrote: post #129
galaxyLogic wrote: post #124

"Hallucinate" is the wrong word when applied to LLMs. Why? Because in common usage that means you are under the influence of some kind of external substance that has entered your blood-stream, or maybe you are under hypnosis, or shamanistic chanting. That is not the case when somebody says "LLM is hallucinating, again!", It is also sensationalist, implying we don't really know what's going on, why is this LLM halluci…

I couldn't agree more. Words matter and this is a complete mismatch between concepts. > HALLUCINATION: a sensory perception (such as a visual image or a sound) that occurs in the absence of an actual external stimulus and usually arises from neurological disturbance [...] [0] When we let a model run free, without input, or with random input, whatever the model imagined it was experiencing (i.e. its internal represent…

Good point. "Hallucination" is about incorrectly perceived input, not incorrect output.

When people hallucinate they don't usually react to it verbally at all, they just observe it. Not the case with LLMs. They just keep on saying something, because they are designed to produce answers.

Post reply on HN