hallucination problem in LLM, been seeing this. Let me know if someone find a fix please
AI hallucinations: Why LLMs make things up (and how to fix it)
141–150 of 257 posts
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#142LLM hallucinations in fact has a positive side effect too, if you are using them for learning some subject; makes you verify their claims, and finding errors in them is very rewarding.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#143Earlier quoted context omitted.
I agree with your point, but I don't think anthropomorphizing LLMs is helpful. They're statistical estimators trained by curve fitting. All generations are equally valid for the training data, objective and architecture. To me it's much clearer to think about it that way versus crude analogies to human brains.
We can't expect end users to understand what "statistical estimators trained by curve fitting" means. That's why we use high level terms like hallucination. Because it's something everyone can understand even if it's not completely accurate.
> Because it's something everyone can understand
But they will understand the wrong thing. Someone unfamiliar with LLMs but familiar with humans will assume, when told that LLMs 'hallucinate', that it's analogous to a human hallucinating, which is dangerously incorrect.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#144Earlier quoted context omitted.
I’ve had multiple occasions where I’ve asked an LLM how to do in Java and it’ll very confidently answer to use . It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting…
Of course the LLM is bullshitting the user. That's precisely its purpose: LLMs are tools that generate comprehensible sounding language based on probability models that describe what words/tokens tend to be found in proximity to each other. An LLM doesn't actually know anything by reference to verifiable, external facts. Sure, LLMs can be used as fancy search engines that index documents and then answer questions by…
Yeah I get that, but at the same time we have AI hype men talking out of both sides of their mouth:
> This model is revolutionary, it knows everything, can answer anything with perfect accuracy!
“It’s fed me bullshit numerous times”
> OF COURSE it’s bullshitting you, don’t you know how LLMs work?
Like how am I supposed to take any of this tech seriously when the LLM is always answering questions as if it had the utmost confidence in what it is spitting out?
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#145> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
What are you taking about, is not artificially deterministic, it is like that by design. We are fortunate that we can use a logic to encode logic and have it for the most part so the same thing given a fix set of antecedents. We even want this in the "real" world, when I turn the wheel left on my car I don't want it turn left only when it feels like it, when that happens we rightly classify it as a failure. We have t…
The fact is that while the tools exist and may be easy to learn, there’s always that nebulous part called creativity, taste, craftsmanship, expertise, or whatever (like designing good software) that’s hard to acquire. Generative AI is good at giving the illusion you can have that part (by piggybacking on the work of everyone). Which is why people get upset when you shatter that illusion by saying that they still don’t have it.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#146LLM hallucinations in fact has a positive side effect too, if you are using them for learning some subject; makes you verify their claims, and finding errors in them is very rewarding.
Why not just read a book where the author is sincerely trying to teach you?
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#147LLM hallucinations in fact has a positive side effect too, if you are using them for learning some subject; makes you verify their claims, and finding errors in them is very rewarding.
Why not just read a book where the author is sincerely trying to teach you?
Not as interactive, not gamified.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#148Earlier quoted context omitted.
At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers. The difference with LLMs is they simply cannot (currently) do the most complex tasks that some humans can, and when they do produce erroneous output, the errors…
> At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers. This is the big one missed by the frequent comments on here wondering whether LLMs are a fad, or claiming in their current state they cannot be used to replace…
> The yardstick is the peformance of humans in the real world on a specific task.
Humans make humans errors, that we can anticipate, recognize, couter, and mitigate. And the rise of deterministic automation was because they help with the parts that are more likely to generate an error. The LLMs strategy always seems like solving a problem that is orthogonal to business objectives, and mainly serves individuals instead.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#149Earlier quoted context omitted.
I’ve had multiple occasions where I’ve asked an LLM how to do in Java and it’ll very confidently answer to use . It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting…
Of course the LLM is bullshitting the user. That's precisely its purpose: LLMs are tools that generate comprehensible sounding language based on probability models that describe what words/tokens tend to be found in proximity to each other. An LLM doesn't actually know anything by reference to verifiable, external facts. Sure, LLMs can be used as fancy search engines that index documents and then answer questions by…
Hilariously, that really does basically define “bullshitting”.