One pollster’s explanation for why the polls got it wrong • Hacker News Board

Re: One pollster’s explanation for why the polls got it wrong

#211

Post by SV_BubbleTime » Tue, Nov 10, 2020, 10:56 PM UTC

zepto wrote: post #205

SV_BubbleTime wrote: post #193

Earlier quoted context omitted.

You can also look at previous votes on initiatives as semi-accurate polling. Michael Bloomberg / Everytown for Gun Safety / Moms Demand Action / The Trace / Mayors Against Illegal Guns which are all the same entity repeatedly pushed out the statistic that "90% of guns owners want to end private gun sales" of course this is totally misleading, but... When it came a vote in Washington state, after outspending the NRA o…

Previous votes generally aren’t semi-accurate. For example ask people to vote on ‘universal background checks’ alone, you’ll probably get a majority voting in favor. If you ask if a gun owner should give their guns to a trusted friend of family member for safekeeping if they feel depressed or suicidal, most people would also say that was a good idea. Of course they wouldn’t realize they had just criminalized it.

Yea, I can agree to that. Let's say then that we can't make determinations, but can tell when something is just definitely not true, like 90% of gun owners definitely do not support ending private sales.

>If you ask if a gun owner should give their guns to a trusted friend of family member for safekeeping if they feel depressed or suicidal, most people would also say that was a good idea. Of course they wouldn’t realize they had just criminalized it.

Man, whole separate topic right there! People not realizing what gun laws say vs mean is so common it seems like it's by design.

Re: One pollster’s explanation for why the polls got it wrong

#212

Post by joshuamorton » Tue, Nov 10, 2020, 10:57 PM UTC

kube-system wrote: post #207

joshuamorton wrote: post #164

Earlier quoted context omitted.

The important thing isn't people lying to pollsters, its people lying to pollsters in a way that's coordinated with a particular demographic or voting block. If 10% of people surveyed, at random, lie to the pollsters and choose the "opposite" candidate out of spite, you'll actually see a sort of reversion to the mean compared to the actual opinions.

Assuming that people are lying at random. I would bet there is a significant correlation with voting habits -- particularly in this election cycle. Why wouldn't someone who holds the view that polls are corrupt and biased have their attitude towards answering them affected by that?

> Why wouldn't someone who holds the view that polls are corrupt and biased have their attitude towards answering them affected by that?

If your opinion was that polls were biased against your candidate, how would that make you answer? Is it more likely to make you answer in favor of your candidate, answer in favor of the other candidate, or go out of your way to participate in polls to "fight back" against the bias?

You can invent a narrative that supports pretty much any activity. Only some of these narratives have any data behind them.

As far as I'm aware the best-supported theory by data isn't that people intentionally lie, but that they simply don't participate in polls because they narrative pushed by republican campaigns is that the polls are always wrong. This is, we'll say, half true.

What I'm interested in, now that the republican talking point is that the actual polls don't matter, since Democrats are stealing the election anyway, is if Republican turnout will similarly decrease.

Re: One pollster’s explanation for why the polls got it wrong

#213

Post by AnimalMuppet » Tue, Nov 10, 2020, 10:58 PM UTC

smnrchrds wrote: post #33

rootusrootus wrote: post #18

Maybe it doesn't really matter much. I mean, sure, we need some amount of polling because it informs decisions such as who gets to participate in televised debates, etc. But other than that, what does the general public get out of polling? It's a spectator sport. In the end, all that matters is the votes, and people should really be encouraged to remember this. I'd think 2016 and 2020 would be a healthy reminder of t…

> what does the general public get out of polling? The laypeople, not much. But for politically active people, not to mention political parties, polls are the best tool to determine where to do what. If you are way ahead in one race but neck and neck in another, it makes sense for parties and activists to focus their efforts and donations to the competitive race. There is never enough resources to focus on every race…

OK, but when some random pollster calls me, what do I get out of it if the parties can allocate their resources more effectively? That's not anywhere on my list of concerns. It lets you campaign more effectively? Great, go bother someone else.

Re: One pollster’s explanation for why the polls got it wrong

#214

Post by matthewbauer » Tue, Nov 10, 2020, 10:58 PM UTC

daxfohl wrote: post #16

ericb wrote: post #9

Has anyone done a study that maps voting machine software used vs. "polling error" in key states?

Before 2016, polls were much more accurate. And paper ballots were the norm. So there's that.

2018 polls were pretty accurate IIRC. Lots of theories going around about how Trump uniquely screws up polling. Not a ton of evidence, but it is really surprising both 2016 and 2020 were off in the same direction. It could happen due to random chance of course. Too bad we probably will never see Trump on the ballot again to get any more data on this phenomenon.

Re: One pollster’s explanation for why the polls got it wrong

#215

Post by mattkrause » Tue, Nov 10, 2020, 11:00 PM UTC

rayiner wrote: post #167

johnnyg wrote: post #80

The book Thinking in Bets points out that predictions are given with a % of certainty. If I say X party will win at Y% and Z party wins, it doesn't mean the model is broken.

But elections aren’t single events, they are 50 different events. 5% of the time the results in Michigan will be outside the error bars. But if the results are outside the error bars in Michigan, Iowa, Wisconsin, Minnesota, etc., and the polls all got it wrong in the same direction for each (overestimated Biden’s support) that’s actually statistically quite unlikely.

Wisconsin, Michigan, and Minnesota aren’t statistically independent though, so if a poll is wrong in one of them, it’s also likely to be wrong in the others too.

Correlated errors are tough!

Re: One pollster’s explanation for why the polls got it wrong

#216

Post by cwkoss » Tue, Nov 10, 2020, 11:00 PM UTC

jerf wrote: post #94

johnnyg wrote: post #80

The book Thinking in Bets points out that predictions are given with a % of certainty. If I say X party will win at Y% and Z party wins, it doesn't mean the model is broken.

On the other hand, this makes the whole operation unfalsifiable in any reasonable time frame. "I always said the candidate had a .1% chance of winning!" Given two major elections in a row where the results were essentially out of the error bars, it is perfectly reasonable to be dropping your confidence on pollsters very hard. Once was perhaps understandable, but twice is getting into "right .01% of the time" territor…

> On the other hand, this makes the whole operation unfalsifiable in any reasonable time frame. "I always said the candidate had a .1% chance of winning!"

I wonder how many people still think Nate Silver has a credible perspective: he's spun the above strategy into a job in unscientific punditry.

Re: One pollster’s explanation for why the polls got it wrong

#217

Post by [deleted] » Tue, Nov 10, 2020, 11:02 PM UTC

bra-ket wrote: post #152

dillondoyle wrote: post #128

Earlier quoted context omitted.

It's less about 'ashamed of their opinions' or the 'shy trump voter.' The article talks about this. The crux of the problem - at least argued by this authored and from what I see working politics backed up generally - is that those Republican / Trump voters do not trust others and are less likely to take the poll in the first place. Those that do take survey generally do not lie about their support. The article gives…

>do not trust others it's hard to trust someone who calls you 'racist' without evidence. It's also hard to trust people who are willing to destroy your life if you're not on their side.

[deleted]

Re: One pollster’s explanation for why the polls got it wrong

#218

Post by cwkoss » Tue, Nov 10, 2020, 11:03 PM UTC

bryanlarsen wrote: post #25

gnusty_gnurc wrote: post #15

It seems obvious to me that there was massive bias in media towards Biden, and they all happily accepted the narrative that Biden would win in a landslide and dismissed anyone saying otherwise as know-nothings. IMO that's tantamount to election interference, if there's near unanimous media coverage for the entire election season of inaccurate polling that assumes a landslide Biden victory. I didn't vote for Biden or…

If the media should be giving equal status and weight to all the parties, what about the Libertarians and the Greens?

I think that would probably be great for the country. At least give them time proportional to their national vote share.

Re: One pollster’s explanation for why the polls got it wrong

#219

Post by rayiner » Tue, Nov 10, 2020, 11:04 PM UTC

mattkrause wrote: post #215

rayiner wrote: post #167

Earlier quoted context omitted.

But elections aren’t single events, they are 50 different events. 5% of the time the results in Michigan will be outside the error bars. But if the results are outside the error bars in Michigan, Iowa, Wisconsin, Minnesota, etc., and the polls all got it wrong in the same direction for each (overestimated Biden’s support) that’s actually statistically quite unlikely.

Wisconsin, Michigan, and Minnesota aren’t statistically independent though, so if a poll is wrong in one of them, it’s also likely to be wrong in the others too. Correlated errors are tough!

Maybe it’s semantics, but to me that means something is wrong with the model. It’s not just a matter of an unlikely result being happening some percentage of the time. There is some hidden factor that could have been adjusted for ahead of time.

Re: One pollster’s explanation for why the polls got it wrong

#220

Post by boomboomsubban » Tue, Nov 10, 2020, 11:05 PM UTC

rayiner wrote: post #116

> I reached out to the smartest polling guy I know: David Shor, an independent data analyst who’s a veteran of the Obama presidential campaigns who formerly operated a massive web-based survey at Civis Analytics before leaving earlier this year Before being fired for pointing out that violent protests alienate people: https://nymag.com/intelligencer/2020/06/case-for-liberalism-... Schor’s firing is a tremendous examp…

So you're defending the person who is trying to explain why polls got it wrong by using polling data to show that their argument belittling the BLM protests was correct.

This is ridiculous on it's own, but also ignores things like how wrong it is to blame the protests for violence often instigated by the police, how dumb it is to trust polling data from the Nixon campaign when we know he was involved in all sorts of disgusting tactics to get elected, and how there is evidence that the protests strongly helped the Democrats.

https://mobile.twitter.com/tbonier/status/132468779601621811...