Lots of folks in these conversations fail to distinguish between LLMs as a technology and "AI Chatbots" as commercial question answering services. Whether false information was expected or not matters to LLM product developers, but in the context of a commercial question-answering tool, it's irrelevant. Hallucinations are bugs that creates= time-wasting zero-value output, at best, and downright harmful output at worst. If you're selling people LLM pattern generator output, they should expect a lot of bullshit. If you're selling people answers to questions, they should expect accurate answers to their questions. If paying users are really expected to assume every answer is bullshit and vet it themselves, that should probably move from the little print to the big print because a lot of people clearly don't get it.
AI hallucinations: Why LLMs make things up (and how to fix it)
41–50 of 257 posts
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#42Is there a way to code an LLM to just say "I don't know" when it is uncertain or reaching some sort of edge?
If it works properly, it would need to say that it doesn't know that it doesn't know, and then where are you?
(Short answer is yes, but it only works for a limited set of things, and that set can be expanded with effort but will always remain limited.)
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#43I've been playing with Qwen's QwQ-32b, and watching this thing's chain of thought is really interesting. In particular, it's pretty good at catching its own mistakes, and at the same time, gives off a "feeling" of someone very uncertain about themselves, trying to verify their answer again and again. Which seems to be the main reason why it can correctly solve puzzles that some much larger models fail. You can still see it occasionally hallucinate things in the CoT, but they are usually quickly caught and discarded.
The only downsides of this approach is that it requires a lot of tokens before the model can ascertain the correctness of its answer, and also that sometimes it just gives up and concludes that the puzzle is unsolvable (although that second part can be mitigated by adding something like "There is definitely a solution, keep trying until you solve it" to the prompt).
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#44> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
A challenge is that it’s not easy to limit hallucinations without also limiting imagination and synthesis.
In humans.
But also apparently in LLMs.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#45I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…
My go-to to show people who don't understand its limitations used to be the old "how many Ms are there in the word 'minimum' or something along those lines, but looks like it's gotten a bit better at that. I just tried it with GPT4o and it gave me the right number, but the wrong placement. In the past it's given it completely wrong: >how many instances of the letter L are in the word parallel The word parallel contai…
Here's QwQ-32 answering the same question. I'm including the entire answer here including the CoT:
> Let's see. The word is "parallel." I need to count how many times the letter "L" appears in it. Okay, spelling it out: P-A-R-A-L-L-E-L. So, starting from the beginning, first letter is P, not L. Then A, not L. R, not L. Another A, still not L. Then L, that's one L. Next letter is L again, so that's two L's. Then E, not L, and another L, making it three L's. Wait, but when I spelled it again, I saw P-A-R-A-L-L-E-L, and I counted three L's. But let me check again. Parallel: P-A-R-A-L-L-E-L. Yes, L appears three times. So, the answer is three.
> But, maybe there's a trick here. Is there any special consideration for the letter L in this word or in general? Sometimes questions like this might be testing for homophones or something, but I don't think so in this case. It's straightforward: count the L's in "parallel."
> Alternatively, perhaps it's a trick with the word itself. Is "parallel" spelled differently sometimes? I don't think so. It's always P-A-R-A-L-L-E-L. So, three L's.
> Wait a minute, is there a possibility that the word is misspelled in the question? If the word was "paralell," that would be four L's, but that's not the correct spelling. So, sticking with three L's.
> Another angle: is the question referring to uppercase or lowercase L? But in the word "parallel," all letters are lowercase unless it's at the beginning of a sentence, but the question just says "the word parallel," which is typically in lowercase. So, doesn't matter.
> Also, considering that "l" and "I" can sometimes be confused, but in this word, it's L's only, no I's.
> So, my final answer is three instances of the letter L in the word "parallel."
> Final Answer: 3
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#46To my understanding, the reason why companies don't mind the hallucinations is the acceptable error rate for a given system. Let's say something hallucinated 25% of the time, but if that's ok, then it's fine for a certain product. If it only hallucinates 5% of the time, it's good enough for even more products and so on. The market will just choose the LLM appropriately depended on the tolerable error rate.
At scale, you are doing the same thing with humans too. LLMs seem to have an error rate similar to humans for the majority of simple, boring tasks, if not even a bit better since they don't get distracted and start copying and pasting their previous answers.
The difference with LLMs is they simply cannot (currently) do the most complex tasks that some humans can, and when they do produce erroneous output, the errors aren't very human-like. We can all understand a cut and paste error so don't hold it against the operator, but making up sources feels like a lie and breeds distrust.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#47> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
LLMs outputs are no more "hallucinations" than my output would be if I were asked to judge a dressage competition.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#48> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced... Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output. What they are is a value judgement we assign to the…
[dead]
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#49Earlier quoted context omitted.
I'm not them, but I think it's a variation of the subset-sum problem If we modify the question to be "sum to 100" (to just seriously reduce the number of example boxes required) then given: | 50 | 20 | 24 | | 7 | 5 | 1 | | 51 | 51 | 51 | the solution would be | [50] | [20] | [24] | | 7 | [ 5] | [ 1] | | 51 | 51 | 51 | | right | down | win | X | right | up | X | X | X
Correct. That but for 1000. You can build your own with any number of online tools. I don't have a link because it's part of a lesson plan set behind a payment on teachers pay teachers.
Do you have a link to any of those online tools that can be used to generate a puzzle of equivalent complexity to the one that you've tested on?
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#50Earlier quoted context omitted.
Calling them hallucinations was a huge mistake.
The term is actually fine. The problem is when it's divorced from the reality of: > in some sense, hallucination is all LLMs do. They are dream machines. If you understand that, then the term "hallucination" makes perfect sense. Note that this in no way invalidates your point, because the term is constantly used and understood without this context. We would have avoided a lot of confusion if we had based it on the ph…
I agree with your point, but I don't think anthropomorphizing LLMs is helpful. They're statistical estimators trained by curve fitting. All generations are equally valid for the training data, objective and architecture. To me it's much clearer to think about it that way versus crude analogies to human brains.