Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What’s the difference between statistics and machine learning?

thestatsgeek.com

Post reply on HN
Print view View original

81–90 of 95 posts

Re: What’s the difference between statistics and machine learning?

#81

Post by Retra »

ACow_Adonis wrote: post #10

Whatever it is, I imagine it's similar to the difference between math, stats, ml, ai, logic, econometrics, actuaries, epidemiology, etc. For the record, I don't consider those intellectually separate fields, but do accept them to be culturally separate, for better or worse... But then I could never understand why physics, biology or chemistry were considered separate fields either... Or psychology, economics, philoso…

They're considered separate fields because they focus on different problems which are amenable to different techniques, leaving their expert practitioners with very different knowledge bases. You're right that it is a cultural distinction, but that doesn't mean it isn't an important or practical one.

Re: What’s the difference between statistics and machine learning?

#82

Post by graycat »

graycat wrote: post #53
alexgmcm wrote: post #52

Earlier quoted context omitted.

There are non-parametric statistical methods too though (such as bootstrap methods) which don't make such assumptions.

Have two calculus teachers, A and B, each with 20 students. Look at the final exam numbers. Put all the numbers in a bucket, stir briskly, draw out 20 numbers (test scores) and average, average the other 20, and get the difference in the averages. Do this many times. Get the empirical distribution of the differences in the averages. Now look at the difference in the actual average for A and B. If this is out in a tai…

Sorry, was way too busy and typed way too fast and had LOTS of typing errors.

Re: What’s the difference between statistics and machine learning?

#83

Post by nitrogen »

Retric wrote: post #80
sweeneyrod wrote: post #78

Earlier quoted context omitted.

> Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains. That's not my experience at all. What kind of ML requires something beyond basic linear algebra? Being comfortable manipulating matrices is certainly harder than plugging data into an sklearn function, but it's also significantl…

The NN complexity is not a question of doing the calculations. Go on a 5x5 board would be a very straightforward game, go in a 19x19 board is not. Similarly, understanding how NN fiction at a nuts and bolts level stops being enough as you build larger networks.

As a curious onlooker to this discussion, could you point to some references for learning the math that applies to emergent behavior in these larger networks?

Re: What’s the difference between statistics and machine learning?

#84

Post by nitrogen »

peatmoss wrote: post #71
lottin wrote: post #28

Inferential statistics is about explaining an observed outcome in terms of its causing factors. Once we have explained it, then we can make predictions. Machine learning skips the explaining part and goes straight to making predictions, without attempting to understand the underlying process that led to the particular outcome. This would be the main difference, in my opinion.

Yes, this is my observation as well. Put another way, statistics is primarily concerned with understanding the mechanisms behind something. Predictive power can and will often be sacrificed if it aids explanatory power or conceptual elegance. By contrast, machine learning is primarily concerned with making the best prediction possible, even if that means sacrificing an understanding of the underlying mechanisms. The…

If you are trying to predict the failure modes of a cat/dog discriminator on arbitrary images, an understanding of catness/dogness is more useful.

Re: What’s the difference between statistics and machine learning?

#85

Post by dorgo »

AstralStorm wrote: post #74
TuringTest wrote: post #64

Earlier quoted context omitted.

> Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. On the other hand, it can work indefinitely if it solves a problem well enough and it is used always on the same kind of problem. Sure, you will need to understand it better to apply it to new markets, but often that kind of research is outside the scope of a single business, and they d…

How do you know it solves the problem, instead of just tricking your measure? Do you know when it doesn't work? Business does not care about rigor, but then terrible things happen when your face recognition system happens to not work for people with dark skin tone. Even more terrible things happen when it detects cats as people, and worse once it's used to link data into a police database. Even pretty bad things happ…

Why don't we do online learning? Each failure is a new training sample.

Why dont't we do active learning? Or adversarial networks, or....

Sure it is easy to fail in machine learning. But it doesn't mean that we need to undstand how and what our models learn. That's the only big advantage of machine learning: the model learns, so I don't have to.

Re: What’s the difference between statistics and machine learning?

#86

Post by dorgo »

Retric wrote: post #80
sweeneyrod wrote: post #78

Earlier quoted context omitted.

> Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains. That's not my experience at all. What kind of ML requires something beyond basic linear algebra? Being comfortable manipulating matrices is certainly harder than plugging data into an sklearn function, but it's also significantl…

The NN complexity is not a question of doing the calculations. Go on a 5x5 board would be a very straightforward game, go in a 19x19 board is not. Similarly, understanding how NN fiction at a nuts and bolts level stops being enough as you build larger networks.

> Go on a 5x5 board would be a very straightforward game

I think it is not a good comparison. A small board calls for simpler algorithms: just try out all possible games and choose the most favorable one. For larger boards this simple approach doesn't work any more and you start looking for something different. But the architecture of NN networks doesn't change at all if you move from small to large scales. Only the number and size of layers increases.

Edit: actually, that's the reason they call it "deep learning". it has more layers..

Re: What’s the difference between statistics and machine learning?

#87

Post by dorgo »

lottin wrote: post #28

Inferential statistics is about explaining an observed outcome in terms of its causing factors. Once we have explained it, then we can make predictions. Machine learning skips the explaining part and goes straight to making predictions, without attempting to understand the underlying process that led to the particular outcome. This would be the main difference, in my opinion.

>machine learning skips the explaining part and goes straight to making predictions

what? Then we should call it machine oracle. It uses magic to make the right predictions without any understanding. It's like seeing a strong AI beating the turing test and saying: it was just lucky. I can think only of two possible ways to make right predictions without understanding: luck and cheating. Sure, sometimes ML cheats.

Re: What’s the difference between statistics and machine learning?

#88

Post by dorgo »

mhermher wrote: post #79
js8 wrote: post #11

That's easy - statisticians take pride in models that are understandable, while machine learning practitioners take pride in models that are not.

Statistics is science and ML is engineering.

I don't know. My former ML profs do a lot of science too...

Re: What’s the difference between statistics and machine learning?

#89

Post by dorgo »

ACow_Adonis wrote: post #20
js8 wrote: post #11

That's easy - statisticians take pride in models that are understandable, while machine learning practitioners take pride in models that are not.

Understandable? Two words: statistical significance :P I've seen university employees who still don't understand what it is and can't explain it...and just about everyone who uses it gets it wrong...

"statistical significance" - it has too many problems. A colleague complained once, that his test kept alternating between statistical significant and not significant back and forth. I know it's not how it is supposed to work, but most people using it don't know (or care) how to use it. My opinion: just don't use this concept at all.

Re: What’s the difference between statistics and machine learning?

#90

Post by beagle3 »

mushufasa wrote: post #21

The classic explanation is Lei Breiman's 'Two Cultures' paper. He was a statistics professor who left for industry, came back, and tried to get academics to adapt industry approaches. The paper is very readable. http://www2.math.uu.se/~thulin/mm/breiman.pdf An oversimplified version may be: Statistics focuses on fitting data to formally understandable models, whereas data science focuses on solving problems -- even i…

Breiman is also responsible for ACE (Alternating Conditional Expectation) which is in many ways magical. https://en.wikipedia.org/wiki/Alternating_conditional_expect...
Post reply on HN