51–60 of 95 posts
That's easy - statisticians take pride in models that are understandable, while machine learning practitioners take pride in models that are not.
>In science one tries to tell people, in such a way as to be understood by everyone, something that no one ever knew before. But in poetry, it's the exact opposite.
I can't speak for all of machine learning, but classic statistics has some probabilistic assumptions and uses those to prove some theorems about the results of how statistical methods manipulate data. E.g., in regression analysis, the assumptions are (i) there really is a linear mode with the variables to be used; (ii) typically the data will not fit the model exactly and instead there are errors; (iii) the errors ar…
I can't speak for all of machine learning, but classic statistics has some probabilistic assumptions and uses those to prove some theorems about the results of how statistical methods manipulate data. E.g., in regression analysis, the assumptions are (i) there really is a linear mode with the variables to be used; (ii) typically the data will not fit the model exactly and instead there are errors; (iii) the errors ar…
There are non-parametric statistical methods too though (such as bootstrap methods) which don't make such assumptions.
Now look at the difference in the actual average for A and B. If this is out in a tail of the empirical distribution, then we reject the null hypothesis that the two teachers are equally good.
For this non-parametric, distribution-free, resampling, two-sample test, to make theorems about it, which should, will likely need at least an independence assumption and likely an i.i.d. (independent, identically distributed) assumption. Else maybe each student of teacher B is an older sibling of a student of teacher A!!!!
In a nutshell, what's going on in statistical hypothesis testing is that we make the null hypothesis, and that gives us enough assumptions, e.g., i.i.d., to calculate the probability of our calculated test statistic, e.g., the difference in the two averages, being way out in a tail. Without some such null hypothesis assumptions, we have no basis on which to reject anything, are not testing anything.
There's chance of getting all twisted out of shape philosophically( here: E.g., I outlined one statistical hypothesis test for the two teachers A and B. Okay, now consider ALL reasonably relevant* hypothesis: Maybe we on the test I outlined, the two teachers look very different, not equal, maybe teacher B better. But in ALL those hypothesis tests, maybe in one of the tests the two teachers look equally good or even teacher A looks better. Now what do we do? That is, there is a suspicion that teacher B looked better ONLY because of the particular test we chose. Maybe there has been some research to clean up this issue.
The classic explanation is Lei Breiman's 'Two Cultures' paper. He was a statistics professor who left for industry, came back, and tried to get academics to adapt industry approaches. The paper is very readable. http://www2.math.uu.se/~thulin/mm/breiman.pdf An oversimplified version may be: Statistics focuses on fitting data to formally understandable models, whereas data science focuses on solving problems -- even i…
(Gated and threshold units.)
The classic explanation is Lei Breiman's 'Two Cultures' paper. He was a statistics professor who left for industry, came back, and tried to get academics to adapt industry approaches. The paper is very readable. http://www2.math.uu.se/~thulin/mm/breiman.pdf An oversimplified version may be: Statistics focuses on fitting data to formally understandable models, whereas data science focuses on solving problems -- even i…
This seems perfectly fair. And I think that historically there was plenty of use for statistics where people didn't care about the formal understanding, so they were doing crude machine learning before the term became widespread. I've sat through lengthy discussions of machine learning exercises, and could not silence the voice in my head, saying: "This is just curve fitting." Fitting data to an arbitrary curve, and…
Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains.
Inferential statistics is about explaining an observed outcome in terms of its causing factors. Once we have explained it, then we can make predictions. Machine learning skips the explaining part and goes straight to making predictions, without attempting to understand the underlying process that led to the particular outcome. This would be the main difference, in my opinion.
statistics is about explaining an observed
outcome in terms of its causing factors.
This may sound a little like trivializing, but don't we have to know what "statistics" are and what "machine learning" is to say anything about the difference(s) between them? Looking at this through even the lens of multinomial logistic regression, or of econometrics generally, I don't think that "statistics draws population inferences from a sample, while machine learning finds generalizable predictive patterns" ev…
Everybody owes it to themselves to read Breiman's Two Cultures[1]: > There are two cultures in the use of statistical modeling to reach conclusions from data. One assumes that the data are generated by a given stochastic data model. The other uses algorithmic models and treats the data mechanism as unknown. The statistical community has been committed to the almost exclusive use of data models. This commitment has le…
Powerful tools like nonlinear polynomial models, OLS, HMM and RBMs were implemented and devised by statisticians. And not from tribe 2. The difference is the data model is explicit but general.
I'm not sure about machine learning specifically, but I heard somewhere that a data scientist is someone who does statistics, on a Mac, in San Francisco.
> What’s the difference between statistics and machine learning?
> About $50k a year.
Inferential statistics is about explaining an observed outcome in terms of its causing factors. Once we have explained it, then we can make predictions. Machine learning skips the explaining part and goes straight to making predictions, without attempting to understand the underlying process that led to the particular outcome. This would be the main difference, in my opinion.
statistics is about explaining an observed outcome in terms of its causing factors. Isn't it the same for ML? When we train a NN to map input factors to the observed outcome, it will set the weights of the factors that do not influence the outcome to 0.
'A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.'
Merriam Webster's definition of statistics:
'a branch of mathematics dealing with the collection, analysis, interpretation, and presentation of masses of numerical data'
So establish a couple of definitions, and take it from there.
I believe such discussions can easily be taken too far: surely the point of a subject is to group together related topics. Who is to say that the set of topics comprising multiple subjects need to be disjoint?