Are ML and Statistics Complementary? [pdf] • Hacker News Board

Re: Are ML and Statistics Complementary? [pdf]

#31

Post by msellout » Sat, Jan 02, 2016, 4:07 AM UTC

p4wnc6 wrote: post #26

msellout wrote: post #25

Earlier quoted context omitted.

You may have formed your generalization about statisticians from a biased sample. Or perhaps you're conflating statistics (ab)users for statisticians. There are far more people who have heard of a t-stat and r-squared than people I would call statistician.

I disagree. Most of the egregious stuff is in published statistics literature, particularly in econometrics, psychology, medicine, and biology, from researchers whose full-time job is to use statistics to solve applied problems ("domain statisticians" if you will). Even if your definition of "statistician" only applied to Wasserman or Gelman types, I'd still say that the machine learning folks of the same level exhib…

I take the narrow view on "statistician". I agree that many if not most scientists are poorly trained in statistics even though all major journals try to throw a veneer of mathematics on their publications.

As for the comparison with ML, I think a large chunk of the ML community aims for (with good reason) evidence of predictive capacity rather than theoretical soundness. Not everyone. I'll grant that a good portion care deeply about theory. Look at the arguments between SVM folks and "Neural" Nets folks.

It comes down to a difference in focus. Statistics cares about causal inference. Machine Learning cares about prediction. Nothing wrong with either, but theiir techniques are sometimes ill-suited for the other purpose.

Re: Are ML and Statistics Complementary? [pdf]

#32

Post by ivansavz » Sat, Jan 02, 2016, 6:21 PM UTC

51109 wrote: post #30

p4wnc6 wrote: post #17

Earlier quoted context omitted.

I have had the exact opposite experience with machine learning and statistics. In my experience, those who come from the 'statistics' side tend to use constructs, like null hypothesis significance testing, which are not consistent even from a theoretical point of view. And further, when they use them, they do awful things like p hacking, or using a direct comparison of t-stats as a model selection criterion, which ar…

Thanks for the links to interesting papers. I really liked the Breiman paper. I did not try to qualify either machine learners and statisticians as bad or good, just pointing out a difference in their approaches to problems. I do not know enough about statistics to make a (negative) quality statement about it. I know a bit more about machine learning though, and there I also see things like: Picking the most favorabl…

Thx for linking to the Caruana video, very interesting.

Re: Are ML and Statistics Complementary? [pdf]

#33

Post by p4wnc6 » Sun, Jan 03, 2016, 5:37 AM UTC

msellout wrote: post #31

p4wnc6 wrote: post #26

Earlier quoted context omitted.

I disagree. Most of the egregious stuff is in published statistics literature, particularly in econometrics, psychology, medicine, and biology, from researchers whose full-time job is to use statistics to solve applied problems ("domain statisticians" if you will). Even if your definition of "statistician" only applied to Wasserman or Gelman types, I'd still say that the machine learning folks of the same level exhib…

I take the narrow view on "statistician". I agree that many if not most scientists are poorly trained in statistics even though all major journals try to throw a veneer of mathematics on their publications. As for the comparison with ML, I think a large chunk of the ML community aims for (with good reason) evidence of predictive capacity rather than theoretical soundness. Not everyone. I'll grant that a good portion…

I agree with your distinction between groups who care about "causal inference" like the debates between Judea Pearl and Andrew Gelman on the role of toy problems in statistics, and groups who care more about "prediction engineering" (as long as we are careful to also admit that people in the ML prediction engineering camp care very, very much about the theoretical properties of their methods, especially in avoiding overfitting, because engineering predicition in a climate of overfitting is useless).

I would just add a big third category that probably encompasses the vast majority of people who "work in statistics" and that would be people who are not interested in causal inference nor in predictive efficacy but are interested in a much less rigorous idea of "explanatory modeling" -- and this group generally is very poor with statistical hygiene.