Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nate Silver Went Against the Grain for Some at The Times

publiceditor.blogs.nytimes.com

Post reply on HN
Print view View original

81–90 of 94 posts

Re: Nate Silver Went Against the Grain for Some at The Times

#81

Post by wtallis »

halo wrote: post #60
chasing wrote: post #4

I feel like traditional pundits got pissed off because he kept wrecking their narratives. And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. Anyway: The New York Times should've worked harder to keep him. They should fi…

>And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. How did he do that? If anything, I would argue the opposite, because his model is proprietary and his explanations (presumably deliberately) vague. 538 is awash with n…

He effectively gave us the equation and kept secret most of the coefficients (though he explained how to choose many of those coefficients). That wouldn't be acceptable for academic purposes, but it seems quite reasonable for journalistic purposes (and is certainly more than any typical political "analyst" provides). He also couldn't give out all the raw data that went into the analyses, because much of it was stuff he had to pay for. Getting past those paywalls and assembling a comprehensive database of polling information is almost certainly the bulk of the work necessary for reproducing his results.

Re: Nate Silver Went Against the Grain for Some at The Times

#82

Post by mattzito »

ebbv wrote: post #18
chasing wrote: post #4

I feel like traditional pundits got pissed off because he kept wrecking their narratives. And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. Anyway: The New York Times should've worked harder to keep him. They should fi…

I couldn't agree more. My respect for the NYT is shaky and them letting Nate Silver leave while keeping the vacuous talking heads you named is just shameful.

The reality is that the NY Times was going to have him focus on politics, with the odd foray onto TV to promote his own brand.

Or, he could sign up with ESPN/Disney, and do sports on ESPN, politics on ABC, and so on.

He'll have a much broader level of impact at ESPN/ABC. I suspect NY Times just couldn't match that.

Re: Nate Silver Went Against the Grain for Some at The Times

#83

Post by mikevm »

halo wrote: post #60
chasing wrote: post #4

I feel like traditional pundits got pissed off because he kept wrecking their narratives. And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. Anyway: The New York Times should've worked harder to keep him. They should fi…

>And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. How did he do that? If anything, I would argue the opposite, because his model is proprietary and his explanations (presumably deliberately) vague. 538 is awash with n…

> 538 is awash with numbers but the algorithms used to generate them are the very definition of "eye-of-newt and other secret ingredients".

I agree. I much prefer the (simpler) work of Sam Wang from the Princeton Election Consortium (http://election.princeton.edu/) who did release MATLAB code of his work and his data sources so you could do it yourself. Somehow, Nate Silver got all the media attention though.

Re: Nate Silver Went Against the Grain for Some at The Times

#84

Post by obstacle1 »

mikevm wrote: post #83
halo wrote: post #60

Earlier quoted context omitted.

>And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. How did he do that? If anything, I would argue the opposite, because his model is proprietary and his explanations (presumably deliberately) vague. 538 is awash with n…

> 538 is awash with numbers but the algorithms used to generate them are the very definition of "eye-of-newt and other secret ingredients". I agree. I much prefer the (simpler) work of Sam Wang from the Princeton Election Consortium ( http://election.princeton.edu/ ) who did release MATLAB code of his work and his data sources so you could do it yourself. Somehow, Nate Silver got all the media attention though.

>Somehow, Nate Silver got all the media attention though.

Yes, somehow the young, geekily attractive popular sports author, whose blog on an extremely popular alternative news site grew a significant cult following, got all the mainstream media attention. It is extremely surprising that said media attention did not go to the duo of crusty, obscure, highfalutin Ivy-league academics instead.

Somehow, that happened. How, we'll never know.

Re: Nate Silver Went Against the Grain for Some at The Times

#85

Post by Yossarian_Lives »

The sports stats on ESPN / Politics on ABC angle is hard to argue against. I like to think though that corporate parent Disney is really looking at the bigger picture here; how Silver fits into its Marvel and Star Wars acquisitions.

Setting aside all creative considerations, the Avengers remains one of Hollywood's most impressive acts of longterm chutzpah, building individual franchises out of B-string characters helmed by way outta left-field director choices and bringing them all together in one film years down the line.

They could, of course, rest on their laurels and churn out a triptych of Avengers movies, each with their orbiting solo films. But Disney thinks bigger than that. Picture the scene in Avengers 3 (after the credits, natch) where Galactus looms into view over Mos Eisley, Hans Solo all wtf. Turns out the next three Star Wars films have all been building to an Avengers/Star Wars merger, dragging in its wake 15 tv series, 45 ancillary films and a trail of comics so large that collectors turn collectivists in order to purchase vast communal warehouses to store their collections.

As the credits roll on Star Wars/Avengers 3, we'd probably forgive Disney if the final card announced that they were following billg into philanthropy. They now account for 10% of US GDP and President Cory Booker is terrified that the gravy train is about to come off the rails and plunge the US economy into recession. BOOM. Audiences gasp. It's a post credits sequence. The camera tracks slowly through a darkened office, over a minimalist, uncluttered desk, towards a high-backed chair that's facing the wall. The phone rings. A hand reaches out to put it on speaker phone. We hear an efficient sounding secretary: "Connecting Mr Fett" and then a pause, the sound of breathing perhaps before Fett says "I'm sorry I failed you". The tension is palpable, as the chair slowly begins to rotate towards the camera. OMG. It's Nate Silver, wizard. "No no, Boba, it's all exactly as I planned it."

Cue the incorporation of 538 into the Star Wars/Avengers franchise, a development freighted so subtly in the accumulated oeuvre to date that only Nate Silver's super algorithm can track and maintain a consistent canon, further fuelling a blog empire.

Re: Nate Silver Went Against the Grain for Some at The Times

#86

Post by malkia »

wtallis wrote: post #81
halo wrote: post #60

Earlier quoted context omitted.

>And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. How did he do that? If anything, I would argue the opposite, because his model is proprietary and his explanations (presumably deliberately) vague. 538 is awash with n…

He effectively gave us the equation and kept secret most of the coefficients (though he explained how to choose many of those coefficients). That wouldn't be acceptable for academic purposes, but it seems quite reasonable for journalistic purposes (and is certainly more than any typical political "analyst" provides). He also couldn't give out all the raw data that went into the analyses, because much of it was stuff…

If only the FICO score algorithm was available somewhere...

Re: Nate Silver Went Against the Grain for Some at The Times

#87

Post by mathattack »

chasing wrote: post #4

I feel like traditional pundits got pissed off because he kept wrecking their narratives. And he upset them further by turning political prognostication into something understandable by common readers -- instead of something only pointy-hatted political wizards could divine with the right amount of eye-of-newt and other secret ingredients. Anyway: The New York Times should've worked harder to keep him. They should fi…

I don't think Friedman and Dowd should be mutually exclusive with him. It's multiple avenues towards the truth - see who gets their first. When he's correct, it's good for science and math.

Re: Nate Silver Went Against the Grain for Some at The Times

#88

Post by foobarqux »

SkyMarshal wrote: post #77
foobarqux wrote: post #70

Earlier quoted context omitted.

I think the added value that any of these forecasters, including Silver, is greatly overstated. When their accuracy is touted it is usually based on the prediction the day before the election. Is that really useful? More importantly you can get the same predictive power by just directly using recent polling data.

>When their accuracy is touted it is usually based on the prediction the day before the election. That's not true in Nate's case at least. One of his biggest wins was debunking some of the BS "narratives" that pundits tried to spring during the election long before election day, the one about "Romney's momentum" in the final weeks of the election being the most memorable one. Linzer also did a postmortem where he loo…

I didn't communicate my point effectively. Let me try again.

I believe strongly in quantitative analysis of election data. I place zero value in punditry, especially from main stream media. What I don't believe is that some of these complicated models, Silver's in particular, are meaningfully superior to predicting the election by trivially applying recent polling data augmenting it perhaps with a simple weighted combination of polls based on recency or sample size.

I take as a given that Silver's model is better than punditry. I am skeptical that it is better than the trivial model which any undergraduate stats student would cook up.

> The problem is, which polling data do you use? They provide a scientific answer to that question. Instead of cherry picking, they use it all, and weight it based on past accuracy and other factors.

What I dispute is whether the "other factors", which is the secret-sauce that allows Silver to give the impression he has a uniquely predictive model, have any real value.

Thanks for the Linzer link. I have to take time to read it carefully but on first glance it again shows one of the things I take issue with: If you are going to claim that a model is accurate you should be asking, compared to what? How can you justify a complex model if you aren't even going to try to show that it is better than some trivial baseline?

Re: Nate Silver Went Against the Grain for Some at The Times

#89

Post by DanBC »

foobarqux wrote: post #88
SkyMarshal wrote: post #77

Earlier quoted context omitted.

>When their accuracy is touted it is usually based on the prediction the day before the election. That's not true in Nate's case at least. One of his biggest wins was debunking some of the BS "narratives" that pundits tried to spring during the election long before election day, the one about "Romney's momentum" in the final weeks of the election being the most memorable one. Linzer also did a postmortem where he loo…

I didn't communicate my point effectively. Let me try again. I believe strongly in quantitative analysis of election data. I place zero value in punditry, especially from main stream media. What I don't believe is that some of these complicated models, Silver's in particular, are meaningfully superior to predicting the election by trivially applying recent polling data augmenting it perhaps with a simple weighted com…

> I take as a given that Silver's model is better than punditry. I am skeptical that it is better than the trivial model which any undergraduate stats student would cook up.

This seems quite easy to test. Has anyone done so yet?

Re: Nate Silver Went Against the Grain for Some at The Times

#90

Post by SkyMarshal »

foobarqux wrote: post #88
SkyMarshal wrote: post #77

Earlier quoted context omitted.

>When their accuracy is touted it is usually based on the prediction the day before the election. That's not true in Nate's case at least. One of his biggest wins was debunking some of the BS "narratives" that pundits tried to spring during the election long before election day, the one about "Romney's momentum" in the final weeks of the election being the most memorable one. Linzer also did a postmortem where he loo…

I didn't communicate my point effectively. Let me try again. I believe strongly in quantitative analysis of election data. I place zero value in punditry, especially from main stream media. What I don't believe is that some of these complicated models, Silver's in particular, are meaningfully superior to predicting the election by trivially applying recent polling data augmenting it perhaps with a simple weighted com…

What would the trivial/baseline model be, just a straight average of all state polls for the last X days?

That would be interesting to know, but also easy to find out.

Post reply on HN