I do feel like we've reached a local maxima with the current state of LLMs, and researchers need to find something completely different to hit a new maxima (whether that is the global maxima or not, we'll know when we hail our new AI overlords).
Overcoming the limits of current LLMs
101–110 of 111 posts
Re: Overcoming the limits of current LLMs
#102I came here thinking I will learn how to make LLMs better. But leaving with more complicated questions:
1. Do I want LLMs to be trained with licensed data, that's arguably well curated. Or, do I want LLM to scrape the web because it is more democratic in opinions?
2. If hallucination is not about training data but how LLM uses that data to extrapolate info that's not directly present in training data - can we teach it this skill to make better choices?
3. It's easy to define good data for facts. How to define good data for subjective topics?
4. For subjective topics, is it better to have separate LLMs trained with each theme of opinions or one big LLM with a mix of all opinions?
5. Is using LLM to improve its own training data truly helpful as the author claims? If yes - is this recursion method better or it's better to use multiple LLMs together?
Dang! If I interview for a position that requires knowledge of AI - every question they ask will be answered with more questions. smh!
Re: Overcoming the limits of current LLMs
#103Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.
I think it's a pretty good name for the phenomenon -- maybe the only problem with the term is that what models are doing is 100% hallucination all the time -- it's just that when the hallucinations are useful we don't call them hallucinations -- so maybe that is a problem with the term (not sure if that's what you are getting at). But there's nothing at all different about what the model is doing between these cases…
They aren't hallucinating in any way comparable to humans, which implies a delusion in perception. You're describing the quality of output by using a word used to describe the quality of input.
Re: Overcoming the limits of current LLMs
#104Earlier quoted context omitted.
> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree that "hallucination" isn't the best word for it... perhaps something like "confabulation" is better.
And we use "hallucination" because in the ancient times when generative AI meant image generation models would "hallucinate" extra fingers etc. The behavior of text models is similar enough that the wording stuck, and it's not all that bad.
That was a misnomer—hallucination refers to perception, not generation. Completely misled an entire generation of people.
Re: Overcoming the limits of current LLMs
#105Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.
"Hallucinations" implies that someone isn't of sound mental state. We can argue forever about what that means for a LLM and whether that's appropriate, but I think it's absolutely the right attitude and approach to be taking toward these things. They simply do not behave like humans of sound minds, and "hallucinations" conveys that in a way that "confabulations" or even "bullshit" does not. (Though "bullshit" isn't b…
Hallucinations implies that they do behave like a human mind. Why else would you use the word if you were not trying to draw this parallel?
Re: Overcoming the limits of current LLMs
#106Earlier quoted context omitted.
How about "dream-reality confusion (DRC)" ?
There is no dream-reality separation in an LLM, or really any conception of dreams or reality, so I don't think the term makes sense. Hallucination works fine to describe the phenomenon. LLMs work by coalescing textual information. LLM hallucinations occur due to faulty or inappropriate coalescence of information, which is similar to what occurs with actual hallucinations.
"Incoherence" seems like a far more natural fit for what you're describing than a human with a sensory delusion or psychosis.
Re: Overcoming the limits of current LLMs
#107Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.
The problem with "incoherent output" is that it isn't describing the phenomenon at all. There have been cases where LLM output has been incoherent, but modern LLM hallucinations are usually coherent and well-contructed, just completely fabricated.
Do you have an example? Your statement seems trivially contradictory—how do you know it's fabricated without incoherence showing you this? Isn't fabrication the entire point of generative ai?
Re: Overcoming the limits of current LLMs
#108Earlier quoted context omitted.
"Hallucinations" implies that someone isn't of sound mental state. We can argue forever about what that means for a LLM and whether that's appropriate, but I think it's absolutely the right attitude and approach to be taking toward these things. They simply do not behave like humans of sound minds, and "hallucinations" conveys that in a way that "confabulations" or even "bullshit" does not. (Though "bullshit" isn't b…
Hallucinations implies that they do behave like a human mind. Why else would you use the word if you were not trying to draw this parallel?
We're stuck with metaphors for human behavior because the way LLMs operate is so alien and counterintuitive, yet similar enough to human behavior, that we haven't yet developed suitable language to describe it. "Hallucination" gets the point across in general terms, at least.
Re: Overcoming the limits of current LLMs
#109Earlier quoted context omitted.
The problem with "incoherent output" is that it isn't describing the phenomenon at all. There have been cases where LLM output has been incoherent, but modern LLM hallucinations are usually coherent and well-contructed, just completely fabricated.
Do you have an example? Your statement seems trivially contradictory—how do you know it's fabricated without incoherence showing you this? Isn't fabrication the entire point of generative ai?
It seems that your insistence upon using incoherence is based upon a misunderstanding of the word. Coherence does not mean to be factual, but to be logically ordered. Also, the word "fabricated" is generally used colloquially to describe made up information without factual basis.
Re: Overcoming the limits of current LLMs
#110Earlier quoted context omitted.
> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree that "hallucination" isn't the best word for it... perhaps something like "confabulation" is better.
And we use "hallucination" because in the ancient times when generative AI meant image generation models would "hallucinate" extra fingers etc. The behavior of text models is similar enough that the wording stuck, and it's not all that bad.
"hallucination" was coined in the context of text generating RNNs. Specifically in this blog post by Karpathy in 2015:
https://karpathy.github.io/2015/05/21/rnn-effectiveness/