Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Differences in link hallucination and source comprehension across different LLM

mikecaulfield.substack.com

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Differences in link hallucination and source comprehension across different LLM

#31

Post by capnrefsmmat »

motorest wrote: post #23
krzat wrote: post #21

Earlier quoted context omitted.

The approach of generating something and then looking for hallucinations is just stupid. To validate the output I have to be an expert. How do I become an expert if rely on LLMs? It's a dead end.

> The approach of generating something and then looking for hallucinations is just stupid. To validate the output I have to be an expert. No. You only need to check for sources, and then verify these sources exist and they support the claims. It's the very definition of "fact". In some cases, all you need to do is check if a URL that was cited does exist.

If the output is interpreting sources rather than just regurgitating quotes from them, you need to exert judgment to verify they support its claims. When the LLM output is about some highly technical subject, it can require expert knowledge just to judge whether the source supports the claims.

Re: Differences in link hallucination and source comprehension across different LLM

#32

Post by sincerecook »

motorest wrote: post #24
nkrisc wrote: post #22

Earlier quoted context omitted.

Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens.

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens. No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

[dead]

Re: Differences in link hallucination and source comprehension across different LLM

#33

Post by vrighter »

motorest wrote: post #23
krzat wrote: post #21

Earlier quoted context omitted.

The approach of generating something and then looking for hallucinations is just stupid. To validate the output I have to be an expert. How do I become an expert if rely on LLMs? It's a dead end.

> The approach of generating something and then looking for hallucinations is just stupid. To validate the output I have to be an expert. No. You only need to check for sources, and then verify these sources exist and they support the claims. It's the very definition of "fact". In some cases, all you need to do is check if a URL that was cited does exist.

"and suport the claims" is doing some *extremely* heavy lifting there.

I can't write a software program, give the source to the greengrocer and expect him to be able to say anything about its quality. Just like I can't really say much about vegetables.

Re: Differences in link hallucination and source comprehension across different LLM

#34

Post by vrighter »

alkonaut wrote: post #27
vanschelven wrote: post #16

Earlier quoted context omitted.

Including literal 404s... As an outsider it has always struck me as absurd that they don't just do the equivalent of wget over all provided sources.

Or why the LLM doesn’t do a lookup into a subset of the training data as a database and reject the output if it seems to be wrong. A billion of the most urls and the entirety of Wikipedia, arkiv and stackoverflow would go a long way.

If that could be done, then we would be using that and skipping the llms entirely

Re: Differences in link hallucination and source comprehension across different LLM

#35

Post by vrighter »

motorest wrote: post #24
nkrisc wrote: post #22

Earlier quoted context omitted.

Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens.

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens. No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

"correct" for an llm means "fits the statistical distributions in the training data"

"correct" for you is "truth that corresponds to the real world"

They are two very different things. The llm's output is, very much, correct. Because it was never meant to mean anything other than similarity of probability distributions.

It's not what you wanted, but that doesn't make it incorrect. You're just under a wrong assumption about what you were asking for. You were asking for something that looks like it could be true. Even if you ask it to not hallucinate, you're just asking it to make it look like it is not hallucinating. Meanwhile you thought you were asking for the actual, real, answer to your question.

Re: Differences in link hallucination and source comprehension across different LLM

#36

Post by simonw »

simonw wrote: post #25

The key thing I got from this article is that the o3 and Claude 4 projects (I'm differentiating from the models here because the harness of tools around them is critical too) are massively ahead of GPT 4.1 and Gemini 2.5 when it comes to fact checking in a way that benefits from search and web usage. The o3 finding matches my own experience: https://simonwillison.net/2025/Apr/21/ai-assisted-search/#o3... Both o3 and…

Products, not projects.

Re: Differences in link hallucination and source comprehension across different LLM

#37

Post by esafak »

motorest wrote: post #4

Taken from the blog: > Why are we talking about “graduate and PhD-level intelligence” in these systems if they can’t find and verify relevant links — even directly after a search? This is my pet peeves, and recently OpenAI's models seem to have become very militant in how they stand by and push their obviously hallucinated sources. I'm talking about hallucinating answers, when pressed to cite sources they also halluc…

This is trivial to overcome by using a REST client to verify the link through MCP, and by caching results it wouldn't even add much latency.

Re: Differences in link hallucination and source comprehension across different LLM

#38

Post by Timwi »

vrighter wrote: post #35
motorest wrote: post #24

Earlier quoted context omitted.

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens. No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

"correct" for an llm means "fits the statistical distributions in the training data" "correct" for you is "truth that corresponds to the real world" They are two very different things. The llm's output is, very much, correct. Because it was never meant to mean anything other than similarity of probability distributions. It's not what you wanted, but that doesn't make it incorrect. You're just under a wrong assumption…

Oh okay, guess all LLMs are just fine then and we don't need to do any further development on them.

Re: Differences in link hallucination and source comprehension across different LLM

#39

Post by vharuck »

vrighter wrote: post #35
motorest wrote: post #24

Earlier quoted context omitted.

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens. No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

"correct" for an llm means "fits the statistical distributions in the training data" "correct" for you is "truth that corresponds to the real world" They are two very different things. The llm's output is, very much, correct. Because it was never meant to mean anything other than similarity of probability distributions. It's not what you wanted, but that doesn't make it incorrect. You're just under a wrong assumption…

Right, the dialogue between the user and the LLM closely resembles documents used in training the LLM. People argue with, lie to, and misunderstand others on the internet. Here's a totally plausible hypothetical forum discussion:

Person A: I believe X.

Person B: Do you have a source for that?

A: Yes, it was shown by blah blah in the paper yada yada.

B: I don't think that study exists. Share a link?

A: [posts a URL]

B: That's not a real paper. The URL doesn't even work!

A: Works on my machine.

---

I've seen those kind of chats so many times online. Know what I haven't seen very often? When person A says "You're right, I made up that article. Let me look again for a real one, and I might change my opinion depending on what it says."

Re: Differences in link hallucination and source comprehension across different LLM

#40

Post by SubiculumCode »

I do wonder about the role of test time compute in the blog post in terms of document understanding. A non reasoning output (or low test time compute setting) might easily misinterpret the text, but reasoning models can second guess, consider multiple objectives in turn, and can right the ship.

I note that Gemini 2.5 has one of the lowest confabulation/hallucination rates according to this benchmark [1], so am surprised by the results in the blog.

Also, I have found link hallucination and output quality improve when you restrict searches to, for example, only pubmed sources, and to provide the source link directly into the text (as opposed to Gemini deep research usual method for citation).

One reason, I think, is that unrestricted search will get the paper, the related blog posts and press releases, weight them as equal (and independent!) sources of a fact, when we know that nuance is lost in the latter, and maybe because it will then spend more test time compute in the quality sources, not the press-releases.

[1]https://github.com/lechmazur/confabulations/

Post reply on HN