Live data from Hacker News

Show HN: Bullshit Detector – agent skills that fact-check videos and articles

github.com

21–30 of 80 posts

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#21

What do you use as “ground truth”? The page says “independent sources”, and I’m sure there’s too many to list, but my question is how are they vetted as being truthful and how are two sources with opposite viewpoints reconciled?

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#22
post #9

Who checks the checkers?

me. I volunteer as ultimate arbiter of Truth.

maybe some community-driven and curated source blacklist could be implemented. It thing it would be too much for you doing that alone :)

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#25

What do you use as “ground truth”? The page says “independent sources”, and I’m sure there’s too many to list, but my question is how are they vetted as being truthful and how are two sources with opposite viewpoints reconciled?

There is no whitelist source. It can't rate sources for truthfulness. When sources conflicts - skill drops verdict to misleading or unverifiable and both are linked. Can't pick a winner at the moment. That's probably the weakest part and still a judgement call for a model.

Maybe you read them already, but wikipedia has a bunch of interesting pages about sourcing and truth like: https://en.wikipedia.org/wiki/Wikipedia:Verifiability,_not_t... and https://en.wikipedia.org/wiki/Wikipedia:Reliable_sources

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#26
post #21

What do you use as “ground truth”? The page says “independent sources”, and I’m sure there’s too many to list, but my question is how are they vetted as being truthful and how are two sources with opposite viewpoints reconciled?

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

Love it, I'll run tests for "credible sources" and will compare the results. Could be a first feature-request implemented!

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#28
post #21

What do you use as “ground truth”? The page says “independent sources”, and I’m sure there’s too many to list, but my question is how are they vetted as being truthful and how are two sources with opposite viewpoints reconciled?

Something I've found surprisingly effective is telling ChatGPT to "use credible sources" - you can then watch its thinking trace and see it do things like ruling out random blogs, considering media publications with a good reputation for fact checking, and double-checking information that seems unlikely.

I don't think random blogs should be ruled out. Just because a blog is random doesn't mean its untrue.

Re: Show HN: Bullshit Detector – agent skills that fact-check videos and articles

#29
post #6

Awesome idea

Thank you! Would love to hear your results if you'll have a chance to try it

I have an AI newsdesk i vibed together because I was losing too much time hunting for facts inside the torrent of misinformation, I'll try to hook it up to the writers
Post reply on HN