Live data from Hacker News

Machine Learning Is the New Statistics

danielmiessler.com

11–20 of 33 posts

Re: Machine Learning Is the New Statistics

#11

The statement "More wisdom potentially gets extracted when you apply Statistics to more (and better) data, but the analysis itself doesn’t improve with better data." simply isn't true. A hierarchical model, for example, is increasingly able to model subgroups and additional levels of hierarchy as more data are added. Penalized regression (or Bayesian regression) is another example -- the model is structurally differe…

Machine Learning is generically defined as a method of data analysis that automates analytical model building.

That's the part that's going to have the impact---automated improvement of the way Statistics is applied to data analysis.

Is that not major enough to be considered and discussed separately?

Re: Machine Learning Is the New Statistics

#12

Machine learning is subset of statistics. The standard text in ML, "The Elements of Statistical Learning" is authored by statistics Professors. Statistics is the new statistics. The rest is marketing bullshit.

Machine learning is subset of statistics. The standard text in ML, "The Elements of Statistical Learning" is authored by statistics Professors.

To be fair, I think "Machine Learning" was an academic marketing term coined by Computer Science departments. It seems that the term "Statistical Learning" was coined in response by Statistics departments. Other similar marketing buzzwords used by various factions (comp sci, stats, actuarial science, industrial engineers, etc) include "Data Science, "Predictive Analytics," "Data Mining," "Knowledge Discovery," "Knowledge Engineering," "Soft Computing," "Artificial Intelligence," "Big Data," "Deep Learning"... To be honest, it's tiresome and troubling to see academic departments invent and adopt overlapping and vacuous marketing buzzwords.

Re: Machine Learning Is the New Statistics

#13

Machine learning is subset of statistics. The standard text in ML, "The Elements of Statistical Learning" is authored by statistics Professors. Statistics is the new statistics. The rest is marketing bullshit.

Statistics as a field typically concerns itself with fitting models to data by determining the distribution of the process that the data was generated from. Machine learning follows a different approach. Read Breiman's "Two Cultures."

http://projecteuclid.org/euclid.ss/1009213726

Re: Machine Learning Is the New Statistics

#14

Machine learning is subset of statistics. The standard text in ML, "The Elements of Statistical Learning" is authored by statistics Professors. Statistics is the new statistics. The rest is marketing bullshit.

I think names for fields are largely defined by social forces behind it not its ideas. Terms like "statistics" are sociological artifacts, not mathematical ones. The difference between stats and machine learning is analogous to say, the difference between Canada and the United States. They're neighbours, separated at birth, share many of the same values but still each have their own personality, worldview, goals and culture.

Walk into a statistics department in a university, and you can feel the differences in culture. It seems to boil down to things about what one considers sacred. Confidence Intervals, Hypothesis Testing and Rigor are things statisticions hold near and dear to their hearts. Want to be a statistics phd? You'd better understand all the differences between convergence in probability and convergence almost surely. And things like Probability Approximate Correct Learning, VC Dimension, Random Forests, and even Deep Learning should sound like heresy to a true, card carrying, statistician.

Another odd cultural difference. Statisticians love citing old papers. You can call a paper about 20 years old "the classical work of" in ML. Stats has a much higher standard for this. The classics go way, way back to a time before I was born. Its gotta be an issue of pride.

But yea, I do feel for the most part, I feel the fields are converging, even if it seems like ML is consuming statistics, not the other way round. Stats majors are publishing in places like NIPS rather than the traditional journals, not the other way round. Just a feeling though. I have no stats to back up this claim.

But there's no point saying "A is a subset of B" unless you want to spice up your happy hour conversations in the pub.

Re: Machine Learning Is the New Statistics

#15

The statement "More wisdom potentially gets extracted when you apply Statistics to more (and better) data, but the analysis itself doesn’t improve with better data." simply isn't true. A hierarchical model, for example, is increasingly able to model subgroups and additional levels of hierarchy as more data are added. Penalized regression (or Bayesian regression) is another example -- the model is structurally differe…

Machine Learning is generically defined as a method of data analysis that automates analytical model building. That's the part that's going to have the impact---automated improvement of the way Statistics is applied to data analysis. Is that not major enough to be considered and discussed separately?

I think the kicker is that most machine learning is incrementing on a single model. Typically one known from statistics before. The weak learner track of combining models almost guess against this, but I think even then it is usually the same shape of model.

So, I actually agree to an extent. Much as computers can be seen as the "next logic". Only, it is such a "builds on" relationship that I think calling it next is dubious.

Re: Machine Learning Is the New Statistics

#16

Machine learning is subset of statistics. The standard text in ML, "The Elements of Statistical Learning" is authored by statistics Professors. Statistics is the new statistics. The rest is marketing bullshit.

Is that the standard? I always thought it was "Pattern Recognition and Machine Learning" by Bishop, but it's not my field.

Re: Machine Learning Is the New Statistics

#17
People that think ML/AI isn't statistics typically haven't studied statistics, or have a marketing agenda. I can tolerate the latter as a fact of life. But the former ... there is often a disturbing lack of statistical understanding in "ML/AI" practitioners at the ground level, even though the vast majority of their tooling is built on basic multivariate statistics. It's rather inevitable give the sudden sex-appeal of the field, but will lead to an 'AI winter' as those folks over promise and under-deliver. Computational statistics, statistical learning, machine learning, pick your term certainly continues to progress as computational horsepower improves. But as another commenter noted, physicists/chemists still self-identify with quantum mechanics even though the computational methods/approximations for molecular dynamics continue to rapidly improve.

Re: Machine Learning Is the New Statistics

#18
While I am not statistically sophisticated to say whether the statement is true or not, it seems that the mathematical machinery is strongly converging, as witnessed by papers like [1], where some of the statistical machinery is being developed on the ML side. It might be historically interesting to note that 20-30 years ago, ML also more evenly spanned work that would be considered AI based -- as including logic or knowledge representation see [2 - 4] as examples.

[1] Variational Inference: A Review for Statisticians, https://arxiv.org/abs/1601.00670

[2] A Summary of Machine Learning Papers from IJCAI-85 http://web.engr.oregonstate.edu/~tgd/publications/mlj-ijcai8...

[3] Chunking in soar: the anatomy of a general learning mechanism, http://repository.cmu.edu/cgi/viewcontent.cgi?article=2552&c...

[4] Explanation-Based Learning: An Alternative View, http://www.cs.utexas.edu/~ml/papers/ebl-mlj-86.pdf

Post reply on HN