Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: I made a platform for journalists to “open source” their fact checking

sourcedfact.com

Post reply on HN
Print view View original

91–100 of 303 posts

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#91

Post by microcolonel »

all2 wrote: post #60
CoryG89 wrote: post #38

Earlier quoted context omitted.

Does it really make sense to call it "reputation" if you're the only one that can see it?

I think so. You could probably pick something better. Its more of a Pavlovian reinforcement than reputation. You see what got you lots of points (what people liked) and you're more apt to do that in the future. By keeping everyone else blind to someone's scores, you don't have as much popularity bias creeping in.

I think it is better to record the facts about people's opinions, than to count reputation points on identities.

For example, as ternary predicates:

1: water is wet

2: identity_1 corroborates 1

3: 2 by_way_of blind_faith

4: identity_2 denies 1

5: 4 by_way_of blind_faith

etc.

You can go as far with this as you want, and the consumer gets to decide what establishes or hurts credibility, and which statements of fact corroborated by what methods and by whom will either count toward or against the credibility of a specific form of statement on a specific topic by a specific set of identities.

For example, you could have predicates describing conflicts of interest, and identify the relevant conflicts of interest by querying for interests in the predicate in question to find what people you trust will say about the interest of individuals and organizations in corroborating or denying a particular claim.

You could also have predicates which show ways in which a statement is controversial in its general form, but is uncontroversial when refined. For example: liquid water can not be wetted with liquid water.

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#92

Post by ghostbrainalpha »

YazIAm wrote: post #82
rootusrootus wrote: post #59

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

This is more a communication standard than a complicated tech solution. A communication standard for fact checks from public primary sources. Coding this wasn't actually that complicated, figuring out a design that enables readers to review the fact checking as easily as possible and enables fact checkers to collaborate easily took a lot of experimentation though. Although even that is still very much a work in progr…

Exactly. I would just appreciate footnotes in the articles I read that gave a detailed explanation of how much fact checking was done.

That way if a story needs to be expanded upon at a later date, the next journalist can dig deeper easily picking up the trail where the previous journalist left off.

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#93

Post by vonnik »

This is really interesting. It reminds me a bit of peer-reviewing journalism via Web annotation: https://web.hypothes.is/

I have a couple comments to make as a former newspaper reporter:

* Many important stories use anonymous sources that are impossible to verify via crowd-sourcing or by linking to official documents. Much of what happens in the world is not publicly documented nor is it widely known (until a reporter publishes a story). That gap between events and any record of those events is one of the structural problems that allow "fake news" to occur.

* Timing matters. Reporters working for the dwindling number of publications that employ fact-checkers or editors have their facts checked before they publish. For that small group of outlets, this diminishes the number of falsehoods they are responsible for unleashing online. Fact-checking a story after its release is useful, but the cat's already out of the bag. The lie is already speeding around the world at the speed of viral outrage.

* As with any platform where better information leads people to make different choices, there are lots of incentives for publications not engage in crowd-sourced fact-checking. Why do people lie or distort the facts on their dating profiles and job applications? Why do companies put forward their most utopian face? They are trying to shape the world with information. And so are media outlets such as Fox News, to name just the most egregious.

* Fact-checking is a lot of work, and can descend into epistemological quibbles. Most people don't have time do engage in it, or even to understand the debate around a given fact. This is why, in the past, we outsourced this work to the editorial staff of the fourth estate. While this would bring more transparency to the process, I am not sure who would have time to take advantage of that transparency, anymore than the normal reader will closely follow the "talk" tab on a Wikipedia article.

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#94

Post by YazIAm »

SamBam wrote: post #61
YazIAm wrote: post #25

Earlier quoted context omitted.

Well first, the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials. Working on this over the last few years though, I've personally fact checked over 500 assertions.

Yes, but that's not the point. That merely shows that we still need to trust someone -- you. Specifically, let's say you support belief A, and I want to add a whole bunch of facts that support belief B. You're still the gatekeeper to decide whether I get to be a contributor. There's no way for someone to know that you're going to approve equally-qualified contributors on different sides of an issue, so they're always…

Indeed, manual vetting is only for alpha. I should reiterate - the point of this type of open fact checking is you can always verify everything for yourself, so you don't need to trust my credentials.

Long term every review, even those that are flagged, will be accessible. Albeit, those that are flagged will be in a separate section, but the point is they will always be accessible, so that you never have to trust me or the community, you can always review for yourself.

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#95

Post by teleclimber »

SamBam wrote: post #68
YazIAm wrote: post #32

Earlier quoted context omitted.

I 100% agree that the reputation system will be they key to scaling this. I'm currently seeing promise with my experiments with reputation scoring. As for "How do you create a platform that allows correct ideas to flourish even when they go against conventional wisdom?" Great question, however my goal here is a bit more modest, it's to create a platform that allows facts which are provable by public primary sources t…

The bots would be a serious issue if this ever became big enough, since they can add to each other's reputation. Reddit probably has thousands of code-hours devoted to this, and bots are still a problem. There is also the problem that humans will probably tend to upvote "facts" that they agree with, and visa-versa. I'm not sure that reputation is the solution. And yet "expert" verification introduces trust. I'm not s…

Yes it's a super tough problem, not to be underestimated by anybody working on such a platform.

If there is any easily automatable way gaining reputation, then the bots will use that, and if you can easily pass reputation too, then the rep system won't hold.

> I'm not sure what the solution is.

Ditto, and I've ben pondering this on and off for years :/

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#96

Post by scott_s »

YazIAm wrote: post #53
scott_s wrote: post #47

Earlier quoted context omitted.

For this to be useful at scale, you will need to not just design this for good-faith users, but also against bad-faith actors. If this catches on, it will become a target of coordinated disinformation campaigns. Have you been thinking about what to do against that?

Indeed, currently all "verified" reviewers are manually vetted. Long term the goal is to have an open reputation score for every fact checker. I'm currently experimenting with different implementations of this, and seeing good promise.

How would the open reputation score mitigate bad-faith actors trying to exploit the system?

For example, I believe HN has a vote-ring detector, but I think there's also a decent amount of manual intervention from the mods. Amazon's reputation system is under constant attack through a variety of means (paying teams of people for fake reviews, going to the length of having real transactions on Amazon between fake-reviewer and the seller buying the review).

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#97

Post by snarf21 »

rootusrootus wrote: post #59

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

Agreed, it is also a question of goals. Journalists can bend "facts" to support the argument they want. Title are 99% click-bait anymore. Sure there are some people who want to present fair and balanced view points but that takes a ton of work and time. Most "journalism" these days is get likes and move on.

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#98

Post by rootusrootus »

microcolonel wrote: post #85
rootusrootus wrote: post #59

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

I think there is a technical solution to this problem though. A structured graph of supporting facts (including an analog of identity for the sources and corroborators, pointing to the edges) is an excellent datastructure for maintaining a record of reasoning and evidence for a statement of fact. There is, of course, the social issue of convincing people that this is important, but the underlying issue is in part a t…

I'm reminded of the oft-quoted phrase "you cannot reason someone out of a position they did not reason themselves into."

I am not trying to disqualify the solution, more like observing that the existence of provable facts has not yet been any sort of antidote for 'fake news' (whether that refers to the real deal, or the position many people take that any news they don't like is fake news).

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#99

Post by tunesmith »

rootusrootus wrote: post #59

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

Why does it seem like that to you? Do you assume that the project should solve the problem entirely in order for it to be of any help or have any value?

Re: Show HN: I made a platform for journalists to “open source” their fact checking

#100

Post by weberc2 »

snarf21 wrote: post #97
rootusrootus wrote: post #59

Great project, but this seems like a classic example of a programmer naively assuming a technical solution can solve a social problem.

Agreed, it is also a question of goals. Journalists can bend "facts" to support the argument they want. Title are 99% click-bait anymore. Sure there are some people who want to present fair and balanced view points but that takes a ton of work and time. Most "journalism" these days is get likes and move on.

Right. Many prominent newsrooms seem to regard fact checking as an obstacle to avoid.

Still, it seems valuable to make it easy for readers to view the supporting or contradictory evidence. That might’ve stymied many notable media hit-pieces. Of course, many newsrooms wouldn’t want this (at least I can’t imagine The Guardian and it’s ilk would want to draw attention to their own impoverished fact:BS ratios), so it would have to be a browser plugin or similar.

Post reply on HN