71–80 of 257 posts
Completely misses the fact that a big part of the reason why llms hallucinate sp much is because there's a huge innate bias towards producing more tokens over just stopping.
The process of thinking for an LLM involves the use of words, which is why prompts that ask the LLM to only return the answer will cause lower quality.
Is there a way to code an LLM to just say "I don't know" when it is uncertain or reaching some sort of edge?
"It" does not know when it does not know. A LLM is a funny old beast that basically outputs words one after another based on probabilities. There is no reasoning as we would know it involved. However, I'll tentatively allow that you do get a sort of "emergent behaviour" from them. You do seem to get some form of intelligent output from a prompt but correctness is not built in, nor is any sort of reasoning. The exampl…
But it does know when it has uncertainty.
In the chatgpt api this is logprobs, each generated token has a level of uncertainty, so:
"2+2="
The next token is with almost 100% certainty 4.
"Today I am feeling"
The next token will be very uncertain, it might be "happy", it might be "sad", it might be all sorts of things.
Earlier quoted context omitted.
LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.
I’ve had multiple occasions where I’ve asked an LLM how to do in Java and it’ll very confidently answer to use . It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting…
I suspect its so hard to get them to say "I don't know" because if they were biased towards responding that way then I would assume thats almost all they would ever say, since "I don't know" is an appropriate answer to every question imaginable.
> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
> The trouble is that we software engineers have spent so long working in an artificially deterministic world that we're not used to designing and evaluating probabilistic quality control systems for computer output. I think that's a mischaracterization and not really accurate. As a trade, we're familiar with probabilistic/non-deterministic components and how to approach them. You were closer when you used quotes aro…
Yes, users.
Earlier quoted context omitted.
"It" does not know when it does not know. A LLM is a funny old beast that basically outputs words one after another based on probabilities. There is no reasoning as we would know it involved. However, I'll tentatively allow that you do get a sort of "emergent behaviour" from them. You do seem to get some form of intelligent output from a prompt but correctness is not built in, nor is any sort of reasoning. The exampl…
"It" does not know when it does not know. But it does know when it has uncertainty. In the chatgpt api this is logprobs, each generated token has a level of uncertainty, so: "2+2=" The next token is with almost 100% certainty 4. "Today I am feeling" The next token will be very uncertain, it might be "happy", it might be "sad", it might be all sorts of things.
By using the word "almost" with regards 2 + 2 = 4, you have not exactly dispelled LLM "nonsense".
A human (with a modicum of maths knowledge) will know that 2 + 2 = 4 (pure integers - a fact by assertion). A maths worrier will get slightly uncomfortable about 2.0 + 2.0 = 4.0 unless they are ensured that decimal places and accuracy are the same thing and a few other things.
A LLM will almost certainly "know" something that is certain, if its training set is conclusive about that. However, it does not know why and if enough of the training set is suitably ambiguous then it (LLM) will drift off course and seem to spout bollocks - "hallucinate".
Earlier quoted context omitted.
I’ve had multiple occasions where I’ve asked an LLM how to do in Java and it’ll very confidently answer to use . It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting…
LLMs don't know things, they just string together responses that are a best fit for what follows from their prompt. I suspect its so hard to get them to say "I don't know" because if they were biased towards responding that way then I would assume thats almost all they would ever say, since "I don't know" is an appropriate answer to every question imaginable.
If I'm beginning with "Once / upon / a" I think the data will show a very high confidence in the word to follow with. So too I would imagine it would know when the trail of breadcrumbs it has been following is of the trashier and low probability kind.
So just tell me. (Or perhaps speak to me and when your confidence is low you can drift into vocal fry territory.)
> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
Calling them hallucinations was a huge mistake.
Earlier quoted context omitted.
We can't expect end users to understand what "statistical estimators trained by curve fitting" means. That's why we use high level terms like hallucination. Because it's something everyone can understand even if it's not completely accurate.
That's a good point. But re: not anthropomorphizing, what's wrong with errors, mistakes or inaccuracies? That's something everybody is familiar with and is more accurate. I'd guess most people have never actually experienced a hallucination anyway, so we're appealing to some vague notion of what that is.
I actually think most people have.
Every time you look at a hot road and see water that mirage is a form of hallucination.
> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.
There are plenty of matters where there is such a source of truth, and LLMs don't know the difference.
Earlier quoted context omitted.
LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.
I’ve had multiple occasions where I’ve asked an LLM how to do in Java and it’ll very confidently answer to use . It would be far more helpful to me to receive an answer like “I don’t think there’s a third party library that does this, you’ll have to write it yourself” than to waste my time telling me a lie. If anything, calling these outputs “hallucinations” is a very polite way of saying that the LLM is bullshitting…
Sure, LLMs can be used as fancy search engines that index documents and then answer questions by referring to them, but even there, the probabilistic nature of the underlying model can still result in mistakes.