Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

FiveThirtyEight has a GitHub repo with story-related data and scripts

github.com

Post reply on HN
Print view View original

11–20 of 38 posts

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#11

Post by smcl »

No story about Nate Silver and 538 is complete without mentioning his new rival in the prediction space - Carl Diggler - who has been kicking his butt on the presidential primaries, calling tough elections correctly where Nate has refused to give more than a "Maybe Sanders, Maybe Clinton..." prediction.

Carl is a work of fiction, invented by a couple of journalists/friends as a parody of pundits: https://www.washingtonpost.com/posteverything/wp/2016/05/09/... - the story is hilarious. Watching this whole thing unfold on Twitter has been a total blast too (check out @carl_diggler), they never break character and have a lot of running gags (such as Carl's troubles in "Family Court"). Here's a wee excerpt from that WP link:

" Carl exists to satirize all that is vacuous, elitist and ridiculous about the media class. From his sycophantic love of candidates in uniform to his hatred of Bernie Bros, from his reverence for “the discourse” to his constant threats of suing the people who troll him on Twitter, Carl is predicated on being myopic, vain and — frankly — wrong.

But something funny happened along the way. Biederman and I, who are neither statisticians nor political scientists, started making educated guesses for our parody about the results of the primaries. And we were right. A lot.

We beat the hacks at their own game by predicting every Democratic winner on Super Tuesday. We told readers who would win in the unpredictable caucuses that FiveThirtyEight didn’t even try to forecast, such as those in Minnesota, Wyoming and even American Samoa. We called 19 out of the past 19 contests. FiveThirtyEight, whose model cannot work without polling, accurately predicted 13 "

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#12

Post by vintermann »

Data is nice, but even with it, 538 has gone alarmingly down the path of clickbait lately. Especially Casselman, with smarmy headlines like "The Rising Unemployment Rate Is Good News" or "Stuck In Your Parents’ Basement? Don’t Blame The Economy".

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#13

Post by InclinedPlane »

smcl wrote: post #11

No story about Nate Silver and 538 is complete without mentioning his new rival in the prediction space - Carl Diggler - who has been kicking his butt on the presidential primaries, calling tough elections correctly where Nate has refused to give more than a "Maybe Sanders, Maybe Clinton..." prediction. Carl is a work of fiction, invented by a couple of journalists/friends as a parody of pundits: https://www.washingt…

538 is the classic "fighting the last war" problem. In 2012 the issue was that polling was unreliable, it was difficult to figure out what was actually going on with the electorate. FiveThirtyEight figured out how to crack that problem, by applying number crunching. This cycle, the problem is that the electorate hasn't actually decided what it wants yet, and there statistical methods aren't helpful at all. There being many examples of 538 "predicting" a 99% probability of a certain victory due to a large margin from polling and then on election day things going completely differently because the electorate changed its mind.

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#14

Post by morgante »

smcl wrote: post #11

No story about Nate Silver and 538 is complete without mentioning his new rival in the prediction space - Carl Diggler - who has been kicking his butt on the presidential primaries, calling tough elections correctly where Nate has refused to give more than a "Maybe Sanders, Maybe Clinton..." prediction. Carl is a work of fiction, invented by a couple of journalists/friends as a parody of pundits: https://www.washingt…

I'm glad you like Carl! The team behind him are super smart and funny (my former coworkers).

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#15

Post by santoshalper »

InclinedPlane wrote: post #13
smcl wrote: post #11

No story about Nate Silver and 538 is complete without mentioning his new rival in the prediction space - Carl Diggler - who has been kicking his butt on the presidential primaries, calling tough elections correctly where Nate has refused to give more than a "Maybe Sanders, Maybe Clinton..." prediction. Carl is a work of fiction, invented by a couple of journalists/friends as a parody of pundits: https://www.washingt…

538 is the classic "fighting the last war" problem. In 2012 the issue was that polling was unreliable, it was difficult to figure out what was actually going on with the electorate. FiveThirtyEight figured out how to crack that problem, by applying number crunching. This cycle, the problem is that the electorate hasn't actually decided what it wants yet, and there statistical methods aren't helpful at all. There bein…

Other than mind reading, what would you propose to solve that problem? It sounds like an intrinsic limit to polling predictions to me.

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#16

Post by InclinedPlane »

santoshalper wrote: post #15
InclinedPlane wrote: post #13

Earlier quoted context omitted.

538 is the classic "fighting the last war" problem. In 2012 the issue was that polling was unreliable, it was difficult to figure out what was actually going on with the electorate. FiveThirtyEight figured out how to crack that problem, by applying number crunching. This cycle, the problem is that the electorate hasn't actually decided what it wants yet, and there statistical methods aren't helpful at all. There bein…

Other than mind reading, what would you propose to solve that problem? It sounds like an intrinsic limit to polling predictions to me.

It's not soluble, even with mind reading, because people change their minds.

The correct solution is just to move on. Too much of the media coverage of elections is about the horse race aspect, almost none of which is actually of positive benefit. Rather than spending so much time trying to guess winners, the media could spend more time and effort actually covering the actual issues of the election in a mature fashion, but it's been decades since they even tried.

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#17

Post by alberto_ol »

InclinedPlane wrote: post #13
smcl wrote: post #11

No story about Nate Silver and 538 is complete without mentioning his new rival in the prediction space - Carl Diggler - who has been kicking his butt on the presidential primaries, calling tough elections correctly where Nate has refused to give more than a "Maybe Sanders, Maybe Clinton..." prediction. Carl is a work of fiction, invented by a couple of journalists/friends as a parody of pundits: https://www.washingt…

538 is the classic "fighting the last war" problem. In 2012 the issue was that polling was unreliable, it was difficult to figure out what was actually going on with the electorate. FiveThirtyEight figured out how to crack that problem, by applying number crunching. This cycle, the problem is that the electorate hasn't actually decided what it wants yet, and there statistical methods aren't helpful at all. There bein…

> There being many examples of 538 "predicting" a 99% probability of a certain victory I think there is only one example of this kind: democratic Michigan primary

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#18

Post by jawns »

I picked a project at random -- the Tarantino one, which lists every death or curse word in every Tarantino movie -- and browsed through the data:

https://github.com/fivethirtyeight/data/blob/master/tarantin...

I found it interesting that the author was willing to catalog each curse word by writing it in its entirety -- except one, the n-word (which, by the way, appears 179 times in Tarantino's movies, with the bulk in "Django Unchained" and "Jackie Brown"). The article for which this data set was compiled similarly censors none but the n-word:

http://fivethirtyeight.com/features/complete-catalog-curses-...

I happen to think that no one of any ethnic group should use the n-word, and I do not use it myself, but I think in the context of cataloging curse words, where you're writing at a sort of meta level (i.e. writing about words, rather than invoking them yourself) it's just as acceptable as in the other cases to write the word out. But the fact that the author made this one exception reveals a bit about how he sees the word in relation to the others.

For instance, you might argue that the varied versions of the f-word might be strong language, but in a different category than the n-word because the latter is an ethnic slur.

But the author has chosen to write out other ethnic slurs in their entirety (e.g. the w-word, referring to Mexicans, and the j-word, referring to Japanese people, and the g-word, related to Asians), and also slurs related to other groups (e.g. the f-word, referring to gay people).

So, even if you're talking about just the set of curse words that insult a particular group of people, the author has set the n-word apart.

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#19

Post by smcl »

morgante wrote: post #14
smcl wrote: post #11

No story about Nate Silver and 538 is complete without mentioning his new rival in the prediction space - Carl Diggler - who has been kicking his butt on the presidential primaries, calling tough elections correctly where Nate has refused to give more than a "Maybe Sanders, Maybe Clinton..." prediction. Carl is a work of fiction, invented by a couple of journalists/friends as a parody of pundits: https://www.washingt…

I'm glad you like Carl! The team behind him are super smart and funny (my former coworkers).

You're lucky - the pair of them seem like really intelligent, funny and creative people

Re: FiveThirtyEight has a GitHub repo with story-related data and scripts

#20

Post by Normal_gaussian »

jawns wrote: post #18

I picked a project at random -- the Tarantino one, which lists every death or curse word in every Tarantino movie -- and browsed through the data: https://github.com/fivethirtyeight/data/blob/master/tarantin... I found it interesting that the author was willing to catalog each curse word by writing it in its entirety -- except one, the n-word (which, by the way, appears 179 times in Tarantino's movies, with the bulk…

n-word -> nigger and f-word -> fuck I understand, but what is the w-word?

I really don't get this obsession over hiding the existence of these words. They obviously don't exist in isolation and are a symptom of a different problem. Perhaps by hiding them people seek to pretend the world has solved these problems?

On top of that, their usage as negative words is only propagated by these shortenings. Words like nigger had (and still do have) neutral and positive connotations [0].

Language is something I deeply believe in, and try to use accurately. People controlling which words I can and can't say really, really, upsets me at level I can't quite find the words express.

[0] https://en.wikipedia.org/wiki/Nigger#Etymology_and_history

Post reply on HN