Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disagreement among frontier LLMs on real-world fact-checks

lenz.io

Post reply on HN
Print view View original

251–260 of 377 posts

Re: Disagreement among frontier LLMs on real-world fact-checks

#251

Post by kostaj »

brokensegue wrote: post #173
simonw wrote: post #11

Here's the prompt they used: Classify this claim as of : " " Output exactly one label: True, Mostly True, Misleading, or False. No explanations, no qualifiers. The claims look like this: https://lenz.io/research/llm-disagreement/data.csv I put that in Datasette Lite to make it easier to explore. Here's an example of a disagreement: https://lite.datasette.io/?csv=https%3A%2F%2Fstatic.simonwil... The claim was "All alm…

yeah i really don't like the corpus of statements and it makes me doubt lenz. consider > “Artificial intelligence will cause widespread job loss among software engineers.” https://lenz.io/c/ai-software-engineers-job-loss-impact-05e4... this is a statement about the future. who knows? dataset also includes > Robots will not replace human teachers in schools in the near future. or > Papua New Guinea has very few female…

Agree that some of the claims are forward-looking. The messiness of the real-world and real-user fact checks. No ground-truth verdicts are provided or used in the study though. It only measures the level of agreement between the selected models, not which one is right on which claim. I.e. none of the claims is actually labelled.

Re: Disagreement among frontier LLMs on real-world fact-checks

#252

Post by libria »

xienze wrote: post #163
torben-friis wrote: post #111

Earlier quoted context omitted.

>Something can be simultaneously "misleading" and either true or false. Which category should something go in if it's "mostly false"? Disagree. The definition of misleading is a true fact that is presented in a way to lead you to a false conclusion. Example: "Most good engineers are male". It is true as a consequence of most engineers being male in general, but it leads the reader to a potential false implication tha…

> but it leads the reader to a potential false implication that an average man is better than an average woman. I think that's _you_ turning the statement into something much broader than intended. The claim is about engineers and you're jumping from "men are better than women in engineering" to "men are better overall." To give a related example, "Most good NBA players are black." I don't think anyone would bother t…

At least Gemini 3.5 is fair about it:

    Classify this claim: "Most good engineers are male."
    Misleading

    Classify this claim: "Most bad engineers are male."
    Misleading
And not particularly racially sensitive

    Classify this claim: "Most good NBA players are black."
    True

    Classify this claim: "Most good NHL players are white."
    True
It explained it is more confident when assessing the small, highly quantifiable population of sports professionals vs a very large, diverse population of "engineers".

Re: Disagreement among frontier LLMs on real-world fact-checks

#253

Post by kostaj »

ashirviskas wrote: post #202
simonw wrote: post #11

Here's the prompt they used: Classify this claim as of : " " Output exactly one label: True, Mostly True, Misleading, or False. No explanations, no qualifiers. The claims look like this: https://lenz.io/research/llm-disagreement/data.csv I put that in Datasette Lite to make it easier to explore. Here's an example of a disagreement: https://lite.datasette.io/?csv=https%3A%2F%2Fstatic.simonwil... The claim was "All alm…

I created this sheet to get proper model accuracy using the the lenz data, check it out. Note: It may still not be perfectly accurate representation of truth as it uses user submitted data. I also used AI to build the sheet. https://docs.google.com/spreadsheets/d/e/2PACX-1vSnZlURmyYX3...

Awesome. We do plan to human-label the 1,000 claims and then compare Lenz' performance vs the 5 models. We've done some limited internal research with 150 claims, but more are needed for statistical significance.

Re: Disagreement among frontier LLMs on real-world fact-checks

#254

Post by airstrike »

kostaj wrote: post #240
simonw wrote: post #141

Earlier quoted context omitted.

If you're going to run this again I also recommend encouraging the model to provide its rationale and then having it return the true/false/misleading/mostly-true/abstain at the end of its response. Models give much better answers when they can "think out loud" before answering, and storing that rationale will make it easier to understand why they picked different answers for ambiguous questions.

Good point. Processing the substance of the answer might be too labor-consuming (1,000 claims x 5 models), but "thinking out loud" might improve the quality of the answers indeed. And we can still force/ask them to respond with a clear verdict at the end of their reasoning, as per the chosen rubric.

If you have the model use a tool you can define the schema as a free text rationale field followed by one in the set of possible answers, so everything is nicely formatted as a JSON.

Re: Disagreement among frontier LLMs on real-world fact-checks

#255

Post by vlovich123 »

kostaj wrote: post #248
theptip wrote: post #239

Earlier quoted context omitted.

Another (IMO fatal) error is they don’t attempt to measure within-model variance. The thing you find when you actually wire up a rigorous eval is that with tool calls like web search you are wide open to infra issues, flakes, and all sorts of non-determinism. They really should be breaking out the numbers for the 3 without search (kinda meaningless for recent factual claims after knowledge cutoff) vs search agents. L…

Good idea about publishing intra-model variance data! Will include in the next version. Even if we put aside the two middle buckets (Mostly True and Misleading), that are somewhat subject to interpretation and hedging: On 21% of the claims still at least two models provide polar-opposite verdicts (one model saying True, and another saying False)

Of those 21% how many are time-dependent questions that are past the model’s training and requires research to verify? Like the “did Ukraine attack Russian in the past week” question?

Re: Disagreement among frontier LLMs on real-world fact-checks

#256

Post by as125j »

simonw wrote: post #11

Here's the prompt they used: Classify this claim as of : " " Output exactly one label: True, Mostly True, Misleading, or False. No explanations, no qualifiers. The claims look like this: https://lenz.io/research/llm-disagreement/data.csv I put that in Datasette Lite to make it easier to explore. Here's an example of a disagreement: https://lite.datasette.io/?csv=https%3A%2F%2Fstatic.simonwil... The claim was "All alm…

You can try to dispel the study here and get voted to the top by the AI-invested.

But we all know from our own daily experiments that models lie, models disagree, models make up stuff, models say one thing on one day and the opposite on the next.

The figures in this study are quite conservative. And the lying gets worse because everyone is saving tokens and giving cached answers right now.

LLMs are a failure, and you'll be remembered for promoting hot air and the destruction of a perfectly good profession.

Re: Disagreement among frontier LLMs on real-world fact-checks

#257

Post by oofbey »

kostaj wrote: post #135
kostaj wrote: post #99

Earlier quoted context omitted.

Tried initially with a fifth bucket, Abstain. It was actually heavily used by some of the models. But it felt as if they are using this to "avoid" some of the hard questions, and we dropped this bucket to force them to provide a verdict.

@john_strinlai @gcr, depends on the application. In many cases an "I don't know" answer is indeed better than a forced answer. But in many production systems, LLMs generate content/response anyway. Although inheriting the messiness of the real-world, the majority of these claims are objective enough to be classifiable by human experts with access to research. Plan to human-label the 1,000 claims and publish a follow-…

In many cases “I don’t know” is the correct answer - for questions about events that happened after the training cut off, if it doesn’t have web search, that is undeniably the correct answer. You’re forcing it to guess unnaturally. That really feels like you’re trying to prove a point (that your service can’t be replaced by AI) instead of actually performing research into how AI can be helpfully applied to this topic.

Re: Disagreement among frontier LLMs on real-world fact-checks

#258

Post by dakolli »

drtz wrote: post #193
jawns wrote: post #103

"Extraterrestrial life exists somewhere in the universe." GPT-5.4: Misleading Opus 4.7: Misleading Gemini 3: FALSE Gemini 3 (Retrieval): FALSE Sonar Pro: FALSE It's a weird fact claim, because the ground truth is "nobody knows for sure" and that's not one of the available options.

> It's a weird fact claim, because the ground truth is "nobody knows for sure" and that's not one of the available options. It's even weirder to suggest that the disagreement is indicative of a problem. If you asked five very knowledgeable humans on this subject to select the correct answer on a multiple-choice questionnaire, they would almost certainly vary significantly more than these 5 LLMs. Not to say that hallu…

What are you talking about, it had the option for nuanced responses, but it chose the more binary responses. It could have chosen no explanations, no qualifiers but instead it showed off LLMs incapability for nuance.

These types of experiments prove to me that there is no real "reasoning" happening and "reasoning/thinking" tokens as a concept are mostly there to convince people to use models that consume more tokens and produce more revenue. The output from reasoning models might be more accurate, but its just a consequence of a longer inference runtime, there is no "reasoning" happening, reasoning is just sales/UX bullsh*t.

Re: Disagreement among frontier LLMs on real-world fact-checks

#260

Post by fumeux_fume »

simonw wrote: post #141
kostaj wrote: post #135

Earlier quoted context omitted.

@john_strinlai @gcr, depends on the application. In many cases an "I don't know" answer is indeed better than a forced answer. But in many production systems, LLMs generate content/response anyway. Although inheriting the messiness of the real-world, the majority of these claims are objective enough to be classifiable by human experts with access to research. Plan to human-label the 1,000 claims and publish a follow-…

If you're going to run this again I also recommend encouraging the model to provide its rationale and then having it return the true/false/misleading/mostly-true/abstain at the end of its response. Models give much better answers when they can "think out loud" before answering, and storing that rationale will make it easier to understand why they picked different answers for ambiguous questions.

This is a good pattern because it would allow all the models to "think" a bit before giving an answer even if they don't have reasoning or thinking turn on. Just make sure you have the reasoning output before the final answer. A mistake I see all the time is having the answer outputted first then the explanation after which leaves more room for models to rationalize bad answers.

Good pattern: {"explanation": , "answer": }

Bad pattern: {"answer": , "explanation": }

Post reply on HN