AI hallucinations: Why LLMs make things up (and how to fix it)
1–10 of 257 posts
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#2the only comment on the prior submission 3 days ago summarizes the whole thing: https://news.ycombinator.com/item?id=42285149
Also, I saw any such blog title as "how to make money in the stock market:" friend, if you knew the answer you wouldn't blog about it you'd be infinitely rich
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#3When people talk about stopping an LLM from "seeing hallucinations instead of the truth", that's like stopping an Ouija-board from "channeling the wrong spirits instead of the right spirits."
It suggests a qualitative difference between desirable and undesirable operation that isn't really there. They're all hallucinations, we just happen to like some of them more than others.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#4I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact.
The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or vertically. The numbers must add up to 1000 when you get to the exit. For people it takes about 5 minutes to figure out. The AI couldn't get it after all 50 students each spent a full 30 minutes changing the prompt to try to get it done. The AI would just randomly add numbers and either add extra at the end to make 1000, or just say the numbers added to 1000 even if it didn't.
The second problem was writing a basic one paragraph essay with one citation. The humans got it done, when with researching for a source, in about 10 minutes. After an additional 30 minutes none of the students could get AI to produce the paragraph without logic or citation errors. It would either make up fake sources, or would just flat out lie about what the sources said. My favorite was a citation related to dairy farming in an essay that was supposed to be about the dangers of smoking tobacco.
This isn't necessarily relevant to the article above, but if there are any teachers here, this is something to do with your students to teach them exactly why not to just use AI for their homework.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#5I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…
My go-to to show people who don't understand its limitations used to be the old "how many Ms are there in the word 'minimum' or something along those lines, but looks like it's gotten a bit better at that. I just tried it with GPT4o and it gave me the right number, but the wrong placement. In the past it's given it completely wrong:
>how many instances of the letter L are in the word parallel
The word parallel contains 3 instances of the letter "L":
The first "L" appears as the fourth letter.
The second "L" appears as the sixth letter.
The third "L" appears as the seventh letter.Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#6When people talk about stopping an LLM from "seeing hallucinations instead of the truth", that's like stopping an Ouija-board from "channeling the wrong spirits instead of the right spirits." It suggests a qualitative difference between desirable and undesirable operation that isn't really there. They're all hallucinations, we just happen to like some of them more than others.
The problem is that LLMs are just convincing enough that people DO trust them which is sort of a problem since AI slop is creeping into everything.
What can be done to solve it (while not perfect) is pretty powerful. You can force feed them the facts (RAG) and then verify the result. Which is way better than trusting LLMs while doing neither of those things (which is what a lot of people do today anyway). See the recent 5 cases of lawyers getting in trouble for ChatGPT hallucinating citations of case law.
LLMs write better than most college students so if you do those two things (RAG + check) you can get college graduate level writing with accurate facts... and that unlocks a bit of value out in the world.
Don't take my word for it look at the proposed valuations of AI companies. Clearly investors think there's something there. The good news is that it hasn't been solved yet so if someone wants to solve it there might be money on the table.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#7Maybe don't make things up in a blog post about LLMs making things up.
Because you don't know how to fix it. Only how to mitigate it.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#8> While the hallucination problem in LLMs is inevitable [0], they can be significantly reduced...
Every article on hallucinations needs to start with this fact until we've hammered that into every "AI Engineer"'s head. Hallucinations are not a bug—they're not a different mode of operation, they're not a logic error. They're not even really a distinct kind of output.
What they are is a value judgement we assign to the output of an LLM program. A "hallucination" is just output from an LLM-based workflow that is not fit for purpose.
This means that all techniques for managing hallucinations (such as the ones described in TFA, which are good) are better understood as techniques for constraining and validating the probabilistic output of an LLM to ensure fitness for purpose—it's a process of quality control, and it should be approached as such. The trouble is that we software engineers have spent so long working in an artificially deterministic world that we're not used to designing and evaluating probabilistic quality control systems for computer output.
[0] They link to this paper: https://arxiv.org/pdf/2401.11817
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#9I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…
My go-to to show people who don't understand its limitations used to be the old "how many Ms are there in the word 'minimum' or something along those lines, but looks like it's gotten a bit better at that. I just tried it with GPT4o and it gave me the right number, but the wrong placement. In the past it's given it completely wrong: >how many instances of the letter L are in the word parallel The word parallel contai…
They probably have a letter counting tool added to it now. that it just knows to call when asked to do this.
you ask it the number of letters and it sends those words off to another tool to count instances of L, but they didn't add a placement one so it's still guessing those.
edit: corrected some typos and phrasing.
Maybe we'll reach a point where the LLM's are just tool calling models and not really giver their own reply.
Re: AI hallucinations: Why LLMs make things up (and how to fix it)
#10I just recently showed a group of college students how and why using AI in school is a bad idea. Telling them it's plagiarism doesn't have an impact, but showing them how it gets even simple things wrong had a HUGE impact. The first problem was a simple numbers problem. It's 2 digit numbers in a series of boxes. You have to add numbers together to make a trail to get from left to right moving only horizontally or ver…
My go-to to show people who don't understand its limitations used to be the old "how many Ms are there in the word 'minimum' or something along those lines, but looks like it's gotten a bit better at that. I just tried it with GPT4o and it gave me the right number, but the wrong placement. In the past it's given it completely wrong: >how many instances of the letter L are in the word parallel The word parallel contai…
I wonder if that is simply a reflection of there being more training data out there about this "problem" and the model hoovering all that up and regurgitating it?