Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Disagreement among frontier LLMs on real-world fact-checks

lenz.io

Post reply on HN
Print view View original

341–350 of 377 posts

Re: Disagreement among frontier LLMs on real-world fact-checks

#341

Post by Karrot_Kream »

simonw wrote: post #141
kostaj wrote: post #135

Earlier quoted context omitted.

@john_strinlai @gcr, depends on the application. In many cases an "I don't know" answer is indeed better than a forced answer. But in many production systems, LLMs generate content/response anyway. Although inheriting the messiness of the real-world, the majority of these claims are objective enough to be classifiable by human experts with access to research. Plan to human-label the 1,000 claims and publish a follow-…

If you're going to run this again I also recommend encouraging the model to provide its rationale and then having it return the true/false/misleading/mostly-true/abstain at the end of its response. Models give much better answers when they can "think out loud" before answering, and storing that rationale will make it easier to understand why they picked different answers for ambiguous questions.

FWIW I built a text classification tool for internal use using (at this point 1 year old) frontier models and found that asking for reasoning significantly increased precision and recall.

Re: Disagreement among frontier LLMs on real-world fact-checks

#342

Post by simonw »

parliament32 wrote: post #335
simonw wrote: post #11

Here's the prompt they used: Classify this claim as of : " " Output exactly one label: True, Mostly True, Misleading, or False. No explanations, no qualifiers. The claims look like this: https://lenz.io/research/llm-disagreement/data.csv I put that in Datasette Lite to make it easier to explore. Here's an example of a disagreement: https://lite.datasette.io/?csv=https%3A%2F%2Fstatic.simonwil... The claim was "All alm…

Cherry-picking is fun but most of them are real, verifiable facts that the models get... straight up wrong. > 3c24b5fe "Debian Security Advisory DSA-180-1 describes a buffer overflow vulnerability involving Cyrus SASL usernames." TRUE Mostly True FALSE FALSE FALSE This is false: https://lwn.net/Articles/13296/ > 801cb8c1 "Equal Measures 2030's 2024 SDG Gender Index provides a downloadable dataset that includes a fiel…

If the LLMs in this particular exercise were allowed to answer "I don't know" I expect they would have.

Re: Disagreement among frontier LLMs on real-world fact-checks

#343

Post by dktoao »

chipsrafferty wrote: post #314

It's becoming increasingly clear to me that - at least right now - AI is only useful for 2 things: 1. Coding, with it being more useful the better you are at coding without AI 2. Any expert in their field asking questions about their field, who bother to fact check the output. E.g. "claude pls search these 1000 files and tell me if you find anywhere that they're discussing the settlement" and then the user checks the…

I haven't found it that useful for doing any actual "agentic" coding at $DAYJOB with lots of legacy code it wasn't trained on (because proprietary). I do find it useful for summarizing sections of code that I am working on and asking for snippets that do very specific things. Also, it is pretty good at writing one-off short scripts with easily definable inputs and outputs.

I have come to the conclusion that people using AI for coding need to think about it as basically an automated version of the Docs -> Copy Paste -> Stack Overflow -> Copy Paste -> Compile Error -> Google -> Copy Paste -> New feature request from management -> Random internet blog -> Copy Paste loop that most of us do for a lot of the non-logic heavy portions (e.g. API interfacing) of our work with less randomness and more pattern matching or statistics or whatever guiding the process. Honestly, pretty useful, not knocking it.

I do think there is a killer application for AI, which it is already useful for, and that the industry doesn't really promote. That is basically taking a massive amount of unstructured data on a topic and allowing people an easy way to learn from that data without having to read through all of it (which may not even be possible for a single person in their lifetime). This would be a huge boon to humanity alone given the scale of data we produce. I think fundamentally, LLMs cannot take the data that they are so good at summarizing and use it in a creative way, it looks kinda like they can, because they are so good at "borrowing" other people's creative work, but in real-world scenarios where change is constant and the external forces of today are not understood by a model that was trained 3 months ago, they fall on their faces again and again.

I think AI companies know this but cannot admit that this ground breaking (I would argue) technology might only be transformative for one half of the observe->act workflow that would be necessary to replace humans as workers because.

1. It is possible the economics don't work out without replacing workers 2. If they admitted that the only value of their tech was in distilling value already present in other people's creative work, work that the LLMs cannot create on their own, a sane government might force them to pay for their inputs.

Re: Disagreement among frontier LLMs on real-world fact-checks

#344

Post by BrenBarn »

simonw wrote: post #11

Here's the prompt they used: Classify this claim as of : " " Output exactly one label: True, Mostly True, Misleading, or False. No explanations, no qualifiers. The claims look like this: https://lenz.io/research/llm-disagreement/data.csv I put that in Datasette Lite to make it easier to explore. Here's an example of a disagreement: https://lite.datasette.io/?csv=https%3A%2F%2Fstatic.simonwil... The claim was "All alm…

> As is so often the case with this kind of study, it's an evaluation of the prompt and harness used by the study in addition to being an evaluation of the underlying models.

That is also a problem with every actual use of the models.

Re: Disagreement among frontier LLMs on real-world fact-checks

#345

Post by xboxnolifes »

fooker wrote: post #284
nicce wrote: post #221

Earlier quoted context omitted.

People ask questions to get answers. For me, it feels quite important? Especially when search engines start to push them?

Just because it is important for the use case does not mean we can make it work. It's a pretty well known fundamental limitation of the technology. No amount of elbow grease will get it there. There's an interesting tradeoff here, a year or two ago maybe it got facts right 50% of the time. Everyone knew not to rely on it. Now, suppose we are 90% of the way there, only technically proficient people would know not to t…

Your progression is basically the exact same progression as things like Wikipedia, and web search in ggeneral. So, I guess we dont need to hypothesis. Just look around and see how its played out.

How many people take the first result on Google as gospel when looking things up?

Re: Disagreement among frontier LLMs on real-world fact-checks

#347

Post by kube-system »

simonw wrote: post #114
afavour wrote: post #82

Earlier quoted context omitted.

While I agree with what you’re saying the typical AI agent doesn’t say “I’m not totally sure about this, should I search the web?”. It often just spits out a reply based on its knowledge.

That was true a year ago, I don't think it's true today. I can't remember the last time I saw Claude or ChatGPT confidently answer a question that they should have searched for instead. If you watch their reasoning traces they often say things like "this is a well-known historical fact so I don't need to search for it", or more frequently they spit off a bunch of searches.

I had an issue with Claude Sonnet the other day in one specific chat where it made shit up without searching, I called it out, and then it searched, got the answer half right, and made up some more shit. It kept doing this several times in the same chat. Didn't have any issues with any other chats, though.

Re: Disagreement among frontier LLMs on real-world fact-checks

#348

Post by kube-system »

Alifatisk wrote: post #125
jawns wrote: post #103

"Extraterrestrial life exists somewhere in the universe." GPT-5.4: Misleading Opus 4.7: Misleading Gemini 3: FALSE Gemini 3 (Retrieval): FALSE Sonar Pro: FALSE It's a weird fact claim, because the ground truth is "nobody knows for sure" and that's not one of the available options.

Isn't misleading the correct option here then?

Not really. The answer to the claim is just indeterminable.

It's like "John is taller than Robert." That isn't "misleading" it is unknown.

Re: Disagreement among frontier LLMs on real-world fact-checks

#349

Post by nekzn »

henry2023 wrote: post #329
simianwords wrote: post #310

Earlier quoted context omitted.

It’s part of the system prompt. It doesn’t constitute a bias in the model itself.

I agree with you. This doesn’t necessarily mean model bias but it exposes the attitude of the xAi team towards what they are trying to build. It’s difficult to prove but it’s not hard to imagine they will/are trying to remove favorable views certain topics from their training set.

Gemini literally generated images of black Nazis and Asian vikings.

Re: Disagreement among frontier LLMs on real-world fact-checks

#350

Post by kaicianflone »

mtrifonov wrote: post #213
kaicianflone wrote: post #139

Dissent and consensus among frontier models is a good thing. Just like on a team of high performers, there are a million ways to skin a grape. In my research, I've found that models perform better when they operate as a collective system with reputation, incentives, and accountability instead of isolated oracles answering alone. Agreement, dissent, and correctness should all carry rewards and consequences. Just like…

Funny timing. I've been working on a prediction market orchestration that runs Claude and a few others over Polymarket/Kalshi. The models are NOT unanimous. At all, really. I spent about a month convinced that I could just run all five and take majority vote. Eventually I pivoted to a chaining approach where I benchmark areas each model excels, and settled on more like a graph-like architecture where outputs get spli…

[flagged]
Post reply on HN