Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Bullshit Detector – agent skills that fact-check videos and articles

github.com

Post reply on HN
Print view View original

61–70 of 80 posts

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#61

Post by BugsJustFindMe »

hunterpayne wrote: post #48
BugsJustFindMe wrote: post #39

Earlier quoted context omitted.

I'm not sure AI helps for that particular task. If you just assume that 100% of everything he says is a lie you'll be 99% correct. Anyone who believes anything he says at this point is living in a constructed reality where facts never matter. Fact checking seems more useful for less pathological cases where it might make a difference.

Cool, now do Fauci... PS I've always assumed all politicians lie, I'm sure you don't think Pelosi or Bush Jr for example were some paragons of truth. Doesn't make it OK, but have some perspective.

> I've always assumed all politicians lie

Everyone lies at various points to some degree or another, ranging from well-intentioned to malignant, and it takes tremendous sleight of hand to pretend that they're even close to comparable.

> Cool, now do [1] Fauci...

[1] Lied that PPE had no value for the public which he admitted later he did to try to preserve limited supplies for healthcare workers. Lied about the possibility of a lab leak having been the origin of COVID19.

> [2] Bush Jr, [3] Pelosi

[2] Lied about Iraq having WMDs and being connected to al-Qaeda to start a war. Built a torture program.

[3] Used insider information to make money. Maybe also knew about the Bush torture program but didn't do anything to stop it if so.

If you had to rank those three, what order would you place them in?

> have some perspective.

Indeed. Perspective: https://en.wikipedia.org/wiki/False_or_misleading_statements...

> Doesn't make it OK, but

Why the "but"? Are you justifying something? The difference between your framework and mine is that my framework statement ends with a period after "Doesn't make it OK."

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#62

Post by stoneman24 »

qntmfred wrote: post #20
N_Lens wrote: post #9

Who checks the checkers?

me. I volunteer as ultimate arbiter of Truth.

To paraphrase Monty Python “supreme executive power derives from a mandate from the masses, not from some farcical aquatic ceremony" (or authority based on a self declaration)

In the age of ai, where you can flood the zone with easily generated content, even a democratic society might struggle to maintain an stable sense of shared reality and history

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#63

Post by qntmfred »

stoneman24 wrote: post #62
qntmfred wrote: post #20

Earlier quoted context omitted.

me. I volunteer as ultimate arbiter of Truth.

To paraphrase Monty Python “supreme executive power derives from a mandate from the masses, not from some farcical aquatic ceremony" (or authority based on a self declaration) In the age of ai, where you can flood the zone with easily generated content, even a democratic society might struggle to maintain an stable sense of shared reality and history

all the more necessary to have trusted curators of information

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#64

Post by skorniienko »

mtweak wrote: post #36
autonomousErwin wrote: post #32

Earlier quoted context omitted.

I could imagine running multiple different AIs seeing if they agree if something is true. I imagine this is similar to how Twitter's/X's community notes work. Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true. So from that approach, you actually want to have two AIs that are on the other side of the spectrum of w…

This is extremely problematic. Much of news is highly syndicated, so what looks like 10 credible sources are actually just 2. I think the best you can aim for is empirical sources: receipts, videos, photo evidence, public disclosures directly at the source. Archiving is ok if disclosed. I don't think LLMs can discern truth when the majority of news sources are incentivized primarily for views.

Filed both issues from this: https://github.com/SerhiiKorniienko/bullshit-detector/issues... - for syndication https://github.com/SerhiiKorniienko/bullshit-detector/issues... - empirical sources

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#65

Post by skorniienko »

simonw wrote: post #21
mysterydip wrote: post #7

What do you use as “ground truth”? The page says “independent sources”, and I’m sure there’s too many to list, but my question is how are they vetted as being truthful and how are two sources with opposite viewpoints reconciled?

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

Added this as issue so I don't forget https://github.com/SerhiiKorniienko/bullshit-detector/issues...

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#66

Post by alealvarezarg »

skorniienko wrote: post #59
alealvarezarg wrote: post #45

I think this would have a much bigger impact as short-form video content such as TikTok, Reels or Shorts. That's where most people consume this kind of content today, not in a .md file. What would really set it apart is adding fact-checking before publishing and letting it post directly to TikTok. That combination would make it something I'd actually use every day.

Not sure that I get your idea about short-form content. It already works with tiktok or yt shorts, especially if it has generated transcription/captions. Otherwise, skill will download video locally and ask your permission to run local models like whisper to transcribe video. Checking before you publish - that's interesting usecase I haven't thought about. Nothing stops you to fact-check your own content and decide w…

[flagged]

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#67

Post by qntmfred »

skorniienko wrote: post #22
qntmfred wrote: post #20

Earlier quoted context omitted.

me. I volunteer as ultimate arbiter of Truth.

maybe some community-driven and curated source blacklist could be implemented. It thing it would be too much for you doing that alone :)

me.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#68

Post by LearnYouALisp »

skorniienko wrote: post #59
alealvarezarg wrote: post #45

I think this would have a much bigger impact as short-form video content such as TikTok, Reels or Shorts. That's where most people consume this kind of content today, not in a .md file. What would really set it apart is adding fact-checking before publishing and letting it post directly to TikTok. That combination would make it something I'd actually use every day.

Not sure that I get your idea about short-form content. It already works with tiktok or yt shorts, especially if it has generated transcription/captions. Otherwise, skill will download video locally and ask your permission to run local models like whisper to transcribe video. Checking before you publish - that's interesting usecase I haven't thought about. Nothing stops you to fact-check your own content and decide w…

[If published] _as_

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#69

Post by LearnYouALisp »

skorniienko wrote: post #52
amelius wrote: post #27

Funny, I recently pasted output from Gemini into Claude, and it said it was total nonsense generated by an overconfident AI. Apparently the "AI-generated bullshit" detection is already part of LLMs.

but have Claude actually fact-checked it or just provided an opinion?

"Lazy" evaluation

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#70

Post by vunderba »

simonw wrote: post #21
mysterydip wrote: post #7

What do you use as “ground truth”? The page says “independent sources”, and I’m sure there’s too many to list, but my question is how are they vetted as being truthful and how are two sources with opposite viewpoints reconciled?

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

For any LLM with web tool usage I’ll often suggest that it pull relevant papers from arXiv, with an expectation that it cite sources. I find that this helps improve accuracy and lets me go deeper into the material if I want to.
Post reply on HN