Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Hallucinating" AIs sound creative, but let's not celebrate being wrong

thereader.mitpress.mit.edu

Post reply on HN
Print view View original

121–130 of 196 posts

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#121

Post by nomel »

0xfae wrote: post #18
ethanbond wrote: post #4

Earlier quoted context omitted.

I've seen more and more of it over the past few weeks. I work in clinical trials and just recently saw the claim that hallucinations can help researchers "expand the search space for molecules they didn't consider." I'm doubtful...

Interesting. If the computer can simulate things that a human wouldn't have thought up, great, but if it's considering things that any sane person knows wouldn't work and passing them off as correct, framing it that way is laughable. That's not brilliant, it's just wrong. If your AI behaves in a way that makes me not trust what it tells me, that's a bug not a feature. But then again, many discoveries have been made b…

> knows wouldn't work and passing them off as correct, framing it that way is laughable.

I don't think this completely true. Considering something that wouldn't work often still leads to ideas, because it kicks your brain of a rut it may be in. This is why it can be incredibly useful to brainstorm with someone that doesn't have expertise in a topic. They'll say zany things that can be inspiring!

Random word sequences are a pretty common way to get inspiration. Something more "on topic" can't be that bad.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#122

Post by happytiger »

pdonis wrote: post #64
jshen wrote: post #44

Earlier quoted context omitted.

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

> humans aren't "optimized" to be correct either Humans at least have a concept of "being correct", even if we don't always set that as our primary goal when communicating. LLMs don't even have a concept of "being correct". That would require having a concept of an "external world" that text refers to, which LLMs don't have. All they have is the text in their training data.

I love this point.

One of the things that has been on my mind of late is the idea of how much of what humans call “correct” or “right” is actually socially generated pressures to conform.

Berns and Asch’s various research on social conformity around what we normally would consider objectively correct answers seems to have some potential applications worth testing in AI land. AIs may be much more performant with AI “social inputs” (and of course all the long term memory improvements currently lacking required to use it as a tool). I am very much excited about the network AI and pipeline work (small teams!) currently coming into vogue. Feels like the “right direction” to make incredible progress.

https://www.psychologytoday.com/us/blog/am-i-right/201404/th...

Otherwise agree with your points exactly. Good comment.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#123

Post by happytiger »

yladiz wrote: post #118
famouswaffles wrote: post #106

Earlier quoted context omitted.

There's plenty support. You just need to know how to read.

Please link to published papers that prove what you’re positing.

To tack on, I would love to read more research on this topic if there is relevant research to read.

I have not read much compelling evidence though I have seen a lot of researchers conjecturing and some inappropriately reaching (and anthropomorphising the living hell out of ai in the process sometimes).

There’s a lot to learn and I don’t think anyone can keep up with all the papers coming out so let’s keep things positive and work together to learn as much as we can og_kalu. Thanks for letting me tag this comment on the end of thread.

This is HN. We’re all nerds here anyways and nobody can keep up with everything in a field moving as fast at AI is these days. :)

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#124

Post by galaxyLogic »

"Hallucinate" is the wrong word when applied to LLMs. Why? Because in common usage that means you are under the influence of some kind of external substance that has entered your blood-stream, or maybe you are under hypnosis, or shamanistic chanting. That is not the case when somebody says "LLM is hallucinating, again!",

It is also sensationalist, implying we don't really know what's going on, why is this LLM hallucinating?

It even sounds like an excuse: Hey my LLM did not come out with reasonable answer, but that is only because it was hallucinating, just like humans sometimes do, so it is even more human-like than we thought. See. Or may it was drunk! That explains it.

No, it's just that LLMs are sometimes on the topic, sometimes not. When somebody says they are "hallucinating" it does not mean they are working in some kind of extra-ordinary mode of operation. They are working just as usual.

What explains what some people (want to) call LLM "hallucinating" is that LLMs sometimes make sense, sometimes they don't.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#125

Post by nomel »

pixl97 wrote: post #69
meowtimemania wrote: post #56

Earlier quoted context omitted.

I think they are seeded. With the same seed, and input, the same model will output the same answer.

I don't think that is exactly true... I've seen arguments based on floating point calculations that accruing errors in FP can generate different results.

I've seen suggestions that it's an implementation problem, in the pipeline, that the same computation, with the same values, and same FP errors (they're deterministic) doesn't result in the same output with a temperature of 0, along with theories that the problem is not chased because it's probably related to optimization/cost reduction.

(trouble finding sources, but it being a bug was from someone involved with OpenAI)

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#127

Post by criddell »

Retric wrote: post #66
simplify wrote: post #52

Earlier quoted context omitted.

No, science is indeed testing random ideas. Limiting what is tested to an arbitrary plausibility score is something funders and sponsors do, not scientists. What you're talking about is ethics, which is important too.

Random ideas include things like: The speed of sound suddenly doubles between 627.500847 to 627.500848 atmospheres. I’ve said it, now try and find someone willing to test it. That’s hard because Scientists want stuff that’s worth spending their limited time on earth looking into.

An AI would probably flag that idea as unlikely and not worth pursuing.

I don’t think we are that far off from having AIs that can read and incorporate the concepts from every paper ever published and start generating random ideas that fit into an internal framework it develops.

I think that within the lifetime of some readers here, an AI will spit out a hypothesis (like a method for producing a room temperature super conductor) that turns out to be true and we will have no idea where it came from. The AI’s knowledge of physics, chemistry, etc… will exceed ours and it trying to explain to us how it works would be like you explaining a singular value decomposition to your dog.

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#129

Post by Nevermark »

galaxyLogic wrote: post #124

"Hallucinate" is the wrong word when applied to LLMs. Why? Because in common usage that means you are under the influence of some kind of external substance that has entered your blood-stream, or maybe you are under hypnosis, or shamanistic chanting. That is not the case when somebody says "LLM is hallucinating, again!", It is also sensationalist, implying we don't really know what's going on, why is this LLM halluci…

I couldn't agree more. Words matter and this is a complete mismatch between concepts.

> HALLUCINATION: a sensory perception (such as a visual image or a sound) that occurs in the absence of an actual external stimulus and usually arises from neurological disturbance [...] [0]

When we let a model run free, without input, or with random input, whatever the model imagined it was experiencing (i.e. its internal representation) would be an hallucination.

> CONFABULATION: to fill in gaps in memory by fabrication

This is exactly the right concept (and therefore word) for what is happening.

[0] https://www.merriam-webster.com/dictionary/hallucination

[1] https://www.merriam-webster.com/dictionary/confabulation

Re: "Hallucinating" AIs sound creative, but let's not celebrate being wrong

#130

Post by happytiger »

danmaz74 wrote: post #68
jshen wrote: post #44

Earlier quoted context omitted.

> "The LLM AI technology generation is optimized to be fluently conversational and not to be factually correct all the time." I always find this point a bit odd because humans aren't "optimized" to be correct either. > "LLMs have no understanding of the underlying reality" I struggle with this one because I see both sides of it. I was making a prompt the other day and gave a CSV file as an input and told the LLM if c…

Humans aren't optimized to be correct, but they are optimized to survive and reproduce themselves in the real world. That's why we develop an understanding of the real world, while LLM can, at most, develop an understanding of human language and the constructs that human language can create. But without the napping that we have between that language and our experience of the world.

It gets very interesting when we start providing digital input of analog source data in film, photograph, sound and speech.

We recently started providing perception inputs, just like you’re talking about.

Obviously it’s nascent and not exactly a revolutionary statement, but it’s interesting to see the progression towards our experience.

I’m also interested in how the analog computing revolution is going to converge with AI in coming years, as we are making huge inroads into analog computing and that means cheap ubiquitous sensor inputs right in time for AI to start maturing.

Post reply on HN