Live data from Hacker News

Show HN: Bullshit Detector – agent skills that fact-check videos and articles

github.com

51–60 of 80 posts

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#51
post #29

Earlier quoted context omitted.

Thank you! Would love to hear your results if you'll have a chance to try it

I have an AI newsdesk i vibed together because I was losing too much time hunting for facts inside the torrent of misinformation, I'll try to hook it up to the writers

Fantastic! Dying to hear the feedback. <3

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#52
post #27

Funny, I recently pasted output from Gemini into Claude, and it said it was total nonsense generated by an overconfident AI. Apparently the "AI-generated bullshit" detection is already part of LLMs.

but have Claude actually fact-checked it or just provided an opinion?

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#54
post #30

I don't even watch most videos anymore. YouTube app -> Share -> Gemini -> Summary. Only if that shows interesting content do I actually watch.

And you are not alone, that is one of the purposes why I am building this

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#55
post #36

Earlier quoted context omitted.

I could imagine running multiple different AIs seeing if they agree if something is true. I imagine this is similar to how Twitter's/X's community notes work. Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true. So from that approach, you actually want to have two AIs that are on the other side of the spectrum of w…

This is extremely problematic. Much of news is highly syndicated, so what looks like 10 credible sources are actually just 2. I think the best you can aim for is empirical sources: receipts, videos, photo evidence, public disclosures directly at the source. Archiving is ok if disclosed. I don't think LLMs can discern truth when the majority of news sources are incentivized primarily for views.

Empirical sources - would be a first candidate to be implemented in a skill. And you are right about 10 credible sources are actually 2. It will look like 10 independent sources to it, real gap. Skill does prefer primary source over secondary. For example it'll prefer NASA article over blog post who cited NASA article.

I'll work on empirical sources improvement in a next releases.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#56

For truth checking actual thinking and logic is required. It doesn't seem to be a proper tool for the task.

True, and it doesn't do logic. It decomposes content into claims and search for every claim citations. It catches a false premise, not a bad inference. Someone with valid reasoning from true facts to a wrong conclusion goes straight through.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#57

I like the idea! How can you let folks know they're being fed nonsense before they even finish the intro?

can't, at least with this design. Report itself takes minutes to proof-check. Realistic version is some kind of extension/plugin idea from earlier in a thread, but I'm not there yet

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#58
post #9

Who checks the checkers?

At the moment - it's model's judgement upon search results for every claim from provided content. If search results returned inaccurate data, or were fabricated - it might affect a verdict for that claim, or highlight that it's unverifiable or contradictory. But there is a room for improvement, what do you suggest? Have a Judge agent which will check results?

[flagged]

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#59

I think this would have a much bigger impact as short-form video content such as TikTok, Reels or Shorts. That's where most people consume this kind of content today, not in a .md file. What would really set it apart is adding fact-checking before publishing and letting it post directly to TikTok. That combination would make it something I'd actually use every day.

Not sure that I get your idea about short-form content. It already works with tiktok or yt shorts, especially if it has generated transcription/captions. Otherwise, skill will download video locally and ask your permission to run local models like whisper to transcribe video.

Checking before you publish - that's interesting usecase I haven't thought about. Nothing stops you to fact-check your own content and decide what to do with it. Point is a report itself not more video content from it

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#60

god i wish these low effort projects were banned from Show HN the entire premise of a "bullshit detector" that's entirely vibe-coded is laughable

why it can't be vibe-coded? Would you be more pleased if it would be poorly hand-coded and claimed "No AI used during development"? What's the point? I don't mind that I've used AI to build this, and I'm open about it ;)

- this is precisely the problem my current project solves

- production grade app generator without AI :)

Post reply on HN