The New York Times Is Eliminating Its Public Editor Role

It wasn't just the New York Times, it was every single news outlet and every single poll. We have been running and studying elections for over 100 years, you would think, by now, that if we had any idea what we were doing, we would have long ago learned how to perform actual predictions using the data available to us.

When people break their legs, we can cure them 100% of the time because medicine has a strong answer for this problem based in thousands of years of medical science. Election polling and prediction is simply not in the category of "solved" problems.

If the statistics were really well understood...as some of the people arguing with you are claiming...WOULDN'T WE HAVE FIGURED THIS OUT BY NOW?

I have had these discussions about polls and statistics with my Liberal friends. Many of them are software engineers. Some of them even work in ML at large scale. They are utterly convinced that the data and statistics just need to be tweaked a little bit more and THEN they will be correct. To quote Donald Trump: WRONG!

There is a simple explanation for what happened that does not get into nit-picking statistical nuances. There is a simple explanation that does not allow people on the other side of the argument to bludgeon less educated opponents into silence by spewing technical jargon about polling data (which is really just a manipulation technique used to silence people by berating their intelligence).

The explanation is that the polling data was simply made up.

Or that many prediction models were created and only the ones favorable to the desired narrative were selected as being "Accurate" by people who are blinded by deep personal political bias.

This outcome scares people who prefer to see the world in a particular way.

People want to believe they aren't biased or that everything they read and see everyday is somehow "clean" or being presented by rational experts. That is a scary thought process...software engineers, in particular, cannot bear to believe that because it violates their underlying belief that the world operates in a logical way.

So instead of thinking like this, they crack open statistics textbooks and start reading chapter 4 at you. Its a defense mechanism to avoid reality.

There are direct financial rewards for telling people exactly what they want to hear. If Nate Silver and 538 consistently told people that Donald Trump would win, it is likely that their page-views would plummet into oblivion. Instead, the vast online hoards who like to be told whatever they want to hear and read news only to have their existing beliefs confirmed swarmed over Nate's websites, clicking on articles and gleefully watching the ads.

What is Nate Silver's punishment for being wrong over and over again? Nothing! There is no identifiable drop-off in his web traffic despite him and his team being wrong about everything this last election.

If what Nate Silver was doing was really "data journalism," shouldn't his following online have dropped off a cliff? It hasn't and it won't because what 538 is doing is not really backed by data and his readers and viewers are not discerning enough to care or believe that what they are reading is just opinion.

I am fairly disgusted by humanity at this point as a result.

Keep strong my brother, you are right. The people quibbling with you are living in an alternate reality where the media isn't incentivized by page views to gravitate towards narratives that result in more clicks. The media is dead.

You don't feel like you are oversimplifying a bit, and applying your own bias? Any pollster knows (and I realize you may ignore this point due to that opening bit) that there are margins for error as long as your sample size is any smaller than 100% of the population. Further, any pollster knows that your data will be skewed by how a question is worded.

The data wasn't made up, and each outcome was within the predicted margin for error. Many people will accuse the statistician of using large margins for error to cover any issues in the data, but that is straight from the opposite end of the spectrum of error.

> People want to believe they aren't biased or that everything they read and see everyday is somehow "clean" or being presented by rational experts. That is a scary thought process...

No one believes that everything they read is clean; anyone who does has (again) oversimplified the world. It isn't black and white. Every day, the margin for error is being examined, polls are being refined, questions are being tweaked. If the pollsters thought they had it all nailed down, why would they even be re-evaluating the method?

The only way to get 100% accurate data would be to poll 100% of the people who are going to vote -- and 0% of the people who aren't. Oddly, that is what happens on voting day -- at great expense, through great manpower, and with great effort. Pollsters simply don't have those kinds of resources.

> There are direct financial rewards for telling people exactly what they want to hear.

People do love to be reinforced, but again you are drastically oversimplifying a more complex issue. No matter what happens, no matter the outcome, people want to read the polls.

I mean, why do we even have them otherwise? So regardless of the error bars or margins, polls are going to keep happening; some will be correct, some won't. Most will fall within fairly narrow margins for error, but often the marker is in that margin -- so the opposite outcome could happen, even within the poll's own admitted rules.

> So instead of thinking like this, they crack open statistics textbooks and start reading chapter 4 at you. Its a defense mechanism to avoid reality.

Unnecessarily inflammatory, I think. The worst part about this rant is that the polls were so close to accurate -- especially if you count the popular vote, instead of the electoral college outcome. It all seems to be a symptom of a country that is, perhaps, too large and diverse for a federal government. I would argue for States' rights here, or say States should be more largely self governing -- but you do need a mechanism to get money from California (taxed $405B, received $343B in federal expenditures) to Kansas (taxed $9.9B, received $24.2B).

That was a bit of a tangent; the point is that the setup of the United States government, both from the State side and Federal side could use some work.

> What is Nate Silver's punishment for being wrong over and over again?

And what is his reward for how often he is right? He works in an industry with a large margin for error. Are you arguing that statistics should basically be abolished? Fivethirtyeight does far more than political polling, mind.

The other thing is relative; Nate Silver was far closer to the mark than any other publication, and his data were more robust. If you are arguing that Nate should lose traffic, every other organization (including those that argued for a Trump win based on similarly faulty statistics) should receive even more loss?

> If what Nate Silver was doing was really "data journalism," shouldn't his following online have dropped off a cliff?

But he was working with the data he had. Fivethirtyeight doesn't even do its own polls; it aggregates data from other polling organizations. Should Nate have made up the data? That is the exact opposite of what you are arguing, and I think your anger is misplaced.

> I am fairly disgusted by humanity at this point as a result.

If a set of polls being off by less than 3% is your threshold for a disgust in humanity, I am afraid your view is narrowed to the point of being useless. There are bigger things to worry about, at least as far as I am concerned.

I just wanted to point that out, in the end.

If I win the lottery, is it misleading for people to state that my odds of winning were one in a million?