Live data from Hacker News

Differences in link hallucination and source comprehension across different LLM

mikecaulfield.substack.com

41–47 of 47 posts

Re: Differences in link hallucination and source comprehension across different LLM

#41

Earlier quoted context omitted.

[flagged]

Are you being paid to post here? They're giving me nothing. Cheapskates.

if this refers to how often i post in bursts, i have wicked insomnia. I really should remove my credentials from the HN app on my phone. if this is what you meant, you're the first person to call me out!

Re: Differences in link hallucination and source comprehension across different LLM

#42
post #29

Earlier quoted context omitted.

Yeah the meaning was what I said until about 2 weeks ago when someone went viral talking about air travel without a cellphone. Do vulgarities often become accepted?

Some of them do, yes. Tone policing has always been around as well, just surprises me to happen in an online forum about "vulgarities".

I apologize if it came off as tone policing. "Raw dog" sounds innocuous, if a bit strange. There was a radio station that had an internet stream out of Topeka Kansas called "Raw Dog Radio" that was a comedy station/stream; as an example - but it was a direct reference to the sexual connotation. For people who don't know, and notice that "raw dogging" is entering zeitgeist, maybe they'll say it because it sounds silly. It was merely a warning about what it means. The "definition shift" really isn't, people are comparing taking a flight without water, cellphones, etc to sex without a condom (Doesn't matter, flew through the air at hundreds of miles per hour, don't care!) The person i replied to was comparing using an LLM without RAG to risky sex.

I'd also avoid saying "So. How are we going to fuck this pig?" in a meeting about infra problems.

Re: Differences in link hallucination and source comprehension across different LLM

#43

Earlier quoted context omitted.

Yeah the meaning was what I said until about 2 weeks ago when someone went viral talking about air travel without a cellphone. Do vulgarities often become accepted?

Are we allowed to say "sex" or "kill"? Or do we have to start s*lf-c*nsoring everything? Anyway, my last search to how to un-alive children processes gave me nothing. I wonder if those m*n pages are actually wr*tten by pr*f*ss*n*ls.

algospeak originated on Tiktok, with all that implies. You couldn't post videos about sexual assault, suicides, murders, any of that directly, so people started self-censoring and using words - unalive maybe came from Deadpool in 2013? I'm not real interested in the actual etymology right now.

and you jest but there's been pushes to change things like "master/slave", "kill", "whitelist/blacklist", and the like to something different. I don't know how much traction, and i may have been tricked by april 1st posts or something.

Re: Differences in link hallucination and source comprehension across different LLM

#44

Earlier quoted context omitted.

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens. No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

"correct" for an llm means "fits the statistical distributions in the training data" "correct" for you is "truth that corresponds to the real world" They are two very different things. The llm's output is, very much, correct. Because it was never meant to mean anything other than similarity of probability distributions. It's not what you wanted, but that doesn't make it incorrect. You're just under a wrong assumption…

Why isn't the LLM under the wrong assumption? So I don't get from my tool what I need and it's still me at fault? I am not yet ready to bow to the AI overlords, sorry.

Re: Differences in link hallucination and source comprehension across different LLM

#45

Earlier quoted context omitted.

> Seems like the LLM is giving correct output if it’s generating a plausible string of tokens in response to your string of tokens. No. If you prompt it to get a response and then you ask it to cite sources, if it outputs broken links that never existed then it clearly failed to deliver correct output.

Seems that's just expecting things that LLMs were not designed for. It's a token producer based on trained weights, it doesn't use any sources. Even if it were "fixed" so that it only generates URLs that exist, it's still incorrect because it did not use any sources so those URLs are not sources.

Then let's face it: LLMs were not designed to give proper answers. Now that we settled this and the emperor is obviously naked, what?

Re: Differences in link hallucination and source comprehension across different LLM

#46

Earlier quoted context omitted.

Or why the LLM doesn’t do a lookup into a subset of the training data as a database and reject the output if it seems to be wrong. A billion of the most urls and the entirety of Wikipedia, arkiv and stackoverflow would go a long way.

If that could be done, then we would be using that and skipping the llms entirely

Can’t see why that couldn’t be done? You save a http request for a ton of the urls.

Re: Differences in link hallucination and source comprehension across different LLM

#47

Earlier quoted context omitted.

If that could be done, then we would be using that and skipping the llms entirely

Can’t see why that couldn’t be done? You save a http request for a ton of the urls.

Because if the llm could tell right from wrong, it wouldn't have to do this in the first place. It's like the bible clainming it's true because the bible says it's true. Circular logic.
Post reply on HN