Live data from Hacker News

Show HN: Bullshit Detector – agent skills that fact-check videos and articles

github.com

41–50 of 80 posts

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#41
post #36

Earlier quoted context omitted.

I could imagine running multiple different AIs seeing if they agree if something is true. I imagine this is similar to how Twitter's/X's community notes work. Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true. So from that approach, you actually want to have two AIs that are on the other side of the spectrum of w…

This is extremely problematic. Much of news is highly syndicated, so what looks like 10 credible sources are actually just 2. I think the best you can aim for is empirical sources: receipts, videos, photo evidence, public disclosures directly at the source. Archiving is ok if disclosed. I don't think LLMs can discern truth when the majority of news sources are incentivized primarily for views.

You may have overlooked this part of the parent's comment which addresses your syndication issue:

> Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true.

Outfits that blindly parrot talking points would not be at odds and thus not be good candidates.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#43
post #21

Earlier quoted context omitted.

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

I don't think random blogs should be ruled out. Just because a blog is random doesn't mean its untrue.

The reputation of the writer matters a lot. Recent models seem to be pretty good at evaluating author credibility as well.

What's dangerous is treating information from an anonymous author as factual. You gotta check that in other ways.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#44
post #36

Earlier quoted context omitted.

I could imagine running multiple different AIs seeing if they agree if something is true. I imagine this is similar to how Twitter's/X's community notes work. Something along the lines of when you have two accounts that disagree traditionally and they agree on something. That's how you know it's likely to be true. So from that approach, you actually want to have two AIs that are on the other side of the spectrum of w…

This is extremely problematic. Much of news is highly syndicated, so what looks like 10 credible sources are actually just 2. I think the best you can aim for is empirical sources: receipts, videos, photo evidence, public disclosures directly at the source. Archiving is ok if disclosed. I don't think LLMs can discern truth when the majority of news sources are incentivized primarily for views.

It also doesn't help that many news sources will knowingly fudge the truth. There are topics about which they are constant sources of misinformation. Seems like depending on the topic (anything science related) you wouldn't want to use news sources at all. Other topics, you probably only want to use news sources. And arranging all of this weighting of sources is going to be difficult and controversial as well. Doesn't mean it can't be done. Doesn't mean it wouldn't be valuable. But there is a lot to it and most of it isn't so much about technology or math as it is about understanding who should and shouldn't be considered authoritative about what topics.

Sounds challenging to me...

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#45
I think this would have a much bigger impact as short-form video content such as TikTok, Reels or Shorts. That's where most people consume this kind of content today, not in a .md file.

What would really set it apart is adding fact-checking before publishing and letting it post directly to TikTok. That combination would make it something I'd actually use every day.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#46

god i wish these low effort projects were banned from Show HN the entire premise of a "bullshit detector" that's entirely vibe-coded is laughable

why it can't be vibe-coded? Would you be more pleased if it would be poorly hand-coded and claimed "No AI used during development"? What's the point? I don't mind that I've used AI to build this, and I'm open about it ;)

[flagged]

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#47

Can someone please use this skill against the claims in its own repo?

Done, fixed 2 misleading lines in README. You can see example report in repo examples. Thanks for that, btw! At least someone questioned it :)

That was an exceptionally fun to read slop :) Cool idea in any case!

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#48
post #16

Washington Post gave up tracking Trump lies last term in 2021 because it became impossible by human hands with 21+ per day and over 30,000 in their database but with "AI" now it's possible not only to do non-stop but in REALTIME you could even just restrict the source of the check to the paper's own reporting the past fifty years * https://www.washingtonpost.com/graphics/politics/trump-claim... I'd like to see that b…

I'm not sure AI helps for that particular task. If you just assume that 100% of everything he says is a lie you'll be 99% correct. Anyone who believes anything he says at this point is living in a constructed reality where facts never matter. Fact checking seems more useful for less pathological cases where it might make a difference.

Cool, now do Fauci...

PS I've always assumed all politicians lie, I'm sure you don't think Pelosi or Bush Jr for example were some paragons of truth. Doesn't make it OK, but have some perspective.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#49

Earlier quoted context omitted.

Bro, you don’t have to outsource your brain for basic communication on the internet. HN’s community has built-in bullshit detectors.

You are right, no excuses. Will have few grammar errors but I'll own them. Cheers!

Sounds like AI reply to me, lmao
Post reply on HN