Live data from Hacker News

Fake News Challenge

fakenewschallenge.org

51–60 of 233 posts

Re: Fake News Challenge

#51
> It should be possible to build a prototype post-facto “truth labeling” system from a “stance detection” system. Such a system would tentatively label a claim or story as true/false based on the stances taken by various news organizations on the topic, weighted by their credibility.

So who decides what organisations are credible? Or orders them by their credibility? Someone has to create the list. Someone with obvious human biases towards one view or another.

Re: Fake News Challenge

#52
post #39

It makes me sad to see yet another tech team go down the road of "machines will help us filter the truth" They will not, and the reason has to do with language. Ludwig Wittgenstein tackled this 100 years ago. The best that machines will do for you is to label something as true or not as if you had consumed the article and decided on your own That is a completely different thing from identifying fake news or truth The…

Wittgenstein wasn't around when a president used Twitter to both control the media and get elected. Maybe news could rely on first principles approach. Quantify things like political stability, information availability, cultural problems, etc. and maybe apply a first principles approach and maybe an algorithm could tell use what the hell is going on at any given moment and maybe give us some insight into what may go…

> Quantify things like political stability, information availability, cultural problems, etc

Good luck. It's a worthy goal, but we are very far away from being able to do this in a rigorous fashion. There is a reason academia has the distinctions of "hard science", "soft science", and "not science". Cultural and political dynamics, by their inherent nature, might never be objectively quantified.

Re: Fake News Challenge

#53
The problem is not "fake news". The problem is people are treating the news like a product in the capitalist paradigm.

If you don't like the way your local bakery makes their bread you go somewhere else. Now if the media reports on something you don't like someone else will tell you what you want to hear.

This issue happens on both sides of the political spectrum BUT it does seem to be a lot worse on the conservative side of things. All the "liberal" news sources like the NYT and The WashPo seem to be doing lots of reporting on Trump supporters, discussing their fears and hopes, concerns and motivations. Even more conservative sources like the WSJ, not including their editorial page, are trying to take a bigger view of the world.

On the other hand my Apple News feed seems to perpetually have a FOX News story. The one this morning was "Is Trump Bashing the new Celeb nude selfie?"

All I know is the people who seem prone to believing the stupidest stories also believe global warming is a hoax and Obama was not born in this country. A significant portion of the electorate has been drinking from the well and cannot be reached.

Re: Fake News Challenge

#56

It seems like Wikipedia would be familiar with a lot of the same challenges that come with identifying fake news articles. Anyone more familiar than I am with Wikipedia care to comment?

Wikipedia has a policy to only accept reliable secondary sources on controversial political articles. What comprises a "reliable source" is loosely defined, and what sources are considered reliable is ultimately decided by the editors in a somewhat per-article basis, with some global definitions which are not centrally listed anywhere but are enforced by senior editors and their admin friends. Because of the systematic bias that Wikipedia suffers from there is strong partisan bias in this selection, which resulted in a number of wiki clones with their own bias (e.g. Conservapedia), forums dedicated to show how bad this is (e.g. /r/wikiinaction) and has - among other things - been leading to a steady decline in the number of editors in the last couple years - everyone eventually gets bullied away from that place by a clique of editors + admins.

Quoting Wikipedia policy itself:

>If Wikipedia had been available around the sixth century B.C., it would have reported the view that the Earth is flat as a fact and without qualification. And it would have reported the views of Eratosthenes (who correctly determined the earth's circumference in 240BC) either as controversial, or a fringe view. Similarly if available in Galileo's time, it would have reported the view that the sun goes round the earth as a fact, and if Galileo had been a Vicipaedia editor, his view would have been rejected as 'originale investigationis'.

Re: Fake News Challenge

#57
Not to sound cliche but I believe the bigger problem in this country is education, which apparently has failed many in this country on both sides of the political spectrum, if the past couple of years are any indication. I guarantee you fix people's problems with English, Math, and History in the classroom, you will have no need for machines or algorithms to filter junk that influences people online.

Re: Fake News Challenge

#58
I think The goal shouldn't be to detect fake news. The goal should be to detect whether a specific claim is true or false. For that, you simply need a site, like a crowd sourced fact checking sites, where people are required to source their arguments precisely, unlike fake news which doesn't. Then you have a resource to turn to, similarly to how fact checkers are used in the face of fake news.

After that, you can build an engine to crawl news articles or have them submitted, and detect the claims being made. Those claims can be either auto-matched to existing ones or create new ones (deduplication) and then the site has a fresh new stream of claims to fact check.

StackOverflow could build this site on their existing engine. If someone here knows them, can you reply and put me in touch?

Re: Fake News Challenge

#59
Their headline is a bit hyperbolic (oh, the irony, given stance detection!) but the FAQ [1] covers what's really going on:

Q: Why did you choose the stance detection task rather than the task of labeling a claim, headline or story True/False, which seems to be what the fake news problem is all about?

A: (...) Our extensive discussions with journalists and fact checkers made it clear both how difficult "truth labeling" of claims really is, and how they'd rather have reliable semi-automated tool to help them in do their job better rather than fully-automated system whose performance will inevitably fall far short of 100% accuracy. (...)

Q: OK, but what does stance detection have to do with detecting fake news?

A: (...) From our discussions with real-life fact checkers, we realized that gathering the relevant background information about a claim or news story, including all sides of the issue, is a critical initial step in a human fact checker's job. One goal of the Fake News Challenge is to push the state-of-the-art in assisting human fact checkers, by helping them quickly gather the information they need to make their assessment.

In particular, a good Stance Detection solution would allow a human fact checker to enter a claim or headline and instantly retrieve the top articles that agree, disagree or discuss the claim/headline in question. They could then look at the arguments for and against the claim, and use their human judgment and reasoning skills to assess the validity of the claim in question. Such a tool would enable human fact checkers to be fast and effective. (...)

This means they're very much aware that 'solving the fake news issue' in a fully-automated way is a folly, so they are instead looking for tools to classify and retrieve corroborating or dissenting reports about the same topic. I feel this approach demonstrates an awareness of the problem, addresses some of the criticisms raised in this thread, and could lead to useful tools and datasets down the road.

[1] http://www.fakenewschallenge.org/#faq

Re: Fake News Challenge

#60

It makes me sad to see yet another tech team go down the road of "machines will help us filter the truth" They will not, and the reason has to do with language. Ludwig Wittgenstein tackled this 100 years ago. The best that machines will do for you is to label something as true or not as if you had consumed the article and decided on your own That is a completely different thing from identifying fake news or truth The…

An anecdote on language that happened to me earlier today: I wrote an email and said I would re-transmit something. I made a typo and wrote re-re-transmit.

"re-transmit" sounds neutral but "re-re-transmit" sounds passive aggressive. This would be very hard for a machine to interpret

Post reply on HN