What’s the difference between statistics and machine learning?
thestatsgeek.com
What’s the difference between statistics and machine learning?
1–10 of 95 posts
Re: What’s the difference between statistics and machine learning?
#2The main difference is not in techniques, but in goals. The primary goal of statistics is to help a human make an informed decision by quantifying uncertainty. That quantification of uncertainty can be used to explain to other humans why the decision was made.
The primary goal of machine learning is to help a machine make better decisions. As long as it gets the "right" answer, the explanations to humans are not as important.
Re: What’s the difference between statistics and machine learning?
#3The main difference is not in techniques, but in goals. The primary goal of statistics is to help a human make an informed decision by quantifying uncertainty. That quantification of uncertainty can be used to explain to other humans why the decision was made. The primary goal of machine learning is to help a machine make better decisions. As long as it gets the "right" answer, the explanations to humans are not as i…
I think that's close, but disagree with the uncertainty quantification bit. Often in both fields, you'll have a proof that "errors" converge at some asymptotic rate. Often in both fields, when you apply that tool to a finite data set, you don't know the error magnitude in this instance. I don't mean exact error, in the sense of 'if I knew the error, I could correct for it.' I mean you know big-O notation error, but not the constant factors.
I do like the rest of the point you make, and it seems to match OP. Inference vs otherwise. Helping people learn/act/decide based on data data vs helping machines learn/act/decide based on data.
Re: What’s the difference between statistics and machine learning?
#4This may sound a little like trivializing, but don't we have to know what "statistics" are and what "machine learning" is to say anything about the difference(s) between them?
Looking at this through even the lens of multinomial logistic regression, or of econometrics generally, I don't think that "statistics draws population inferences from a sample, while machine learning finds generalizable predictive patterns" even makes sense as a difference. Any prediction is an inference about the population of future events, or of contemporary events not present in the sample. You plug 1000s of events each described by 100 columns into a logistic regression, and you're hoping to get something predictive out of it. Further, as nice as the idea is that you can tease out "factors" from your 100 columns, you don't have to look at "3.3375905e-5 x (spent five years before age 18 in a smoker's home)" for very long to wonder how much 'explanation' you're getting out of the terms in the exponents of your probability functions.
I still can't resist tweaking ML enthusiasts and data scientists: Statistics is what people who know what they're doing are doing. Machine learning is the rest!
Re: What’s the difference between statistics and machine learning?
#5In short:
Machine Learning - making predictions
Statistics - distilling huge amount of data into a few indicators
Machine Learning uses algorithms, optimization/operations research, differential calculus, probability and statistics as it sees fit.
Re: What’s the difference between statistics and machine learning?
#6Reminded me of this [1] by Prof. Rob Tibshirani.
[1]: https://statweb.stanford.edu/~tibs/stat315a/glossary.pdf
Re: What’s the difference between statistics and machine learning?
#7In short: Machine Learning - making predictions Statistics - distilling huge amount of data into a few indicators Machine Learning uses algorithms, optimization/operations research, differential calculus, probability and statistics as it sees fit.
Statistics makes predictions too. Indicators are usually predictions - a mean is called an expected value for a reason.
I find the spectacle of machine-learning practitioners and statisticians furiously using different language to agree with each other quite amusing.
Re: What’s the difference between statistics and machine learning?
#8Everything's a function, right?
Re: What’s the difference between statistics and machine learning?
#9This may sound a little like trivializing, but don't we have to know what "statistics" are and what "machine learning" is to say anything about the difference(s) between them? Looking at this through even the lens of multinomial logistic regression, or of econometrics generally, I don't think that "statistics draws population inferences from a sample, while machine learning finds generalizable predictive patterns" ev…
Everybody owes it to themselves to read Breiman's Two Cultures[1]:
> There are two cultures in the use of statistical modeling to reach conclusions from data. One assumes that the data are generated by a given stochastic data model. The other uses algorithmic models and treats the data mechanism as unknown. The statistical community has been committed to the almost exclusive use of data models. This commitment has led to irrelevant theory, questionable conclusions, and has kept statisticians from working on a large range of interesting current problems. Algorithmic modeling, both in theory and practice, has developed rapidly in fields outside statistics. It can be used both on large complex data sets and as a more accurate and informative alternative to data modeling on smaller data sets. If our goal as a field is to use data to solve problems, then we need to move away from exclusive dependence on data models and adopt a more diverse set of tools.
The difference between inference and prediction can be illustrated with something like a decision tree or a random forest. Statistical inference is "the theory, methods, and practice of forming judgments about the parameters of a population and the reliability of statistical relationships, typically on the basis of random sampling." If you look at something like linear regressions, it makes a lot of assumptions about the data[2]: the distribution of residual errors is normal, there's no multicollinearity, etc.
Random forest don't care. Random forest is a set of steps. You follow the steps, you get an answer. You made no assumptions about your data, about the distribution of it, any of it. You just followed an algorithm.
Algorithms are powerful! Not everything needs to be an inference problem. We're better off when we have lots of tools. You don't have to choose _between_ the two camps. But the "Everything is Statistics!" and the "Everything is Machine Learning!" points of view rob us of ways of thinking about our tools that helps us understand what those tools are.
1) https://projecteuclid.org/download/pdf_1/euclid.ss/100921372...
2) https://thestatsgeek.com/2013/08/07/assumptions-for-linear-r...
Re: What’s the difference between statistics and machine learning?
#10Whatever it is, I imagine it's similar to the difference between math, stats, ml, ai, logic, econometrics, actuaries, epidemiology, etc.
For the record, I don't consider those intellectually separate fields, but do accept them to be culturally separate, for better or worse...
But then I could never understand why physics, biology or chemistry were considered separate fields either...
Or psychology, economics, philosophy, etc etc etc...