Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What’s the difference between statistics and machine learning?

thestatsgeek.com

Post reply on HN
Print view View original

71–80 of 95 posts

Re: What’s the difference between statistics and machine learning?

#71

Post by peatmoss »

lottin wrote: post #28

Inferential statistics is about explaining an observed outcome in terms of its causing factors. Once we have explained it, then we can make predictions. Machine learning skips the explaining part and goes straight to making predictions, without attempting to understand the underlying process that led to the particular outcome. This would be the main difference, in my opinion.

Yes, this is my observation as well. Put another way, statistics is primarily concerned with understanding the mechanisms behind something. Predictive power can and will often be sacrificed if it aids explanatory power or conceptual elegance.

By contrast, machine learning is primarily concerned with making the best prediction possible, even if that means sacrificing an understanding of the underlying mechanisms.

The Venn diagram of methods can have a fair degree of overlap.

Neither disposition is right or wrong, but they tend to have natural places where each makes more sense. If you’re trying to predict whether a picture is of a cat or a dog, you probably don’t care much about the constituent contribution of factors to one pictures dogness or catness. On the other hand, if you’re trying to predict traffic collisions based on characteristics of a roadway, you’re probably less concerned with the predicted number of crashes and more concerned with the relative contribution of a handful of independent variables.

Re: What’s the difference between statistics and machine learning?

#72

Post by candiodari »

Statistics is a set of theories and methods that can be successfully executed on any list of numbers and provably provides mathematically valid predictions under a set of conditions that never occurs in the real world.

This means, more precisely, statistics only works on data series that obey the law of large numbers, and combining/using 2 or more statistical predictions nearly always requires total independence of the predictions and their inputs, which is never the case (for one thing, they always occur on the same planet). Furthermore, the reason people make statistical predictions is to change the outcome, but doing anything to change the outcome always invalidates the statistical method used to collect the data. There are a couple of things statistics never does. Used correctly, it can never predict extreme values. It can never correctly predict values in systems that are too complex, where too many independent variables determine the outome. And "too many" is something like 50 to 500. It can never correctly be used to verify if a deliberate change worked.

Machine learning is much the same, except it never provides mathematically valid predictions.

Despite this, it should probably be mentioned that both do provide useful results, occasionally getting things very, very wrong.

https://xkcd.com/605/

Re: What’s the difference between statistics and machine learning?

#73

Post by didibus »

Statistics + Computer Science = Machine Learning?

I think to me that's the case. But also, Machine Learning describes a computer science goal, to have a computer that learns certain or ideally all parts of its algorithm on its own, either from example or by experimentation.

It just so happens that some of the ideas from statistics lend themselves to help computer science implement such machines that can learn. One can imagine techniques for Machine Learning being discovered in the future which leverage ideas from other fields apart from statistics.

Re: What’s the difference between statistics and machine learning?

#74

Post by AstralStorm »

TuringTest wrote: post #64
AstralStorm wrote: post #55

Earlier quoted context omitted.

Formal understanding is critical to actually know the limitations of any given system. Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains.

> Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. On the other hand, it can work indefinitely if it solves a problem well enough and it is used always on the same kind of problem. Sure, you will need to understand it better to apply it to new markets, but often that kind of research is outside the scope of a single business, and they d…

How do you know it solves the problem, instead of just tricking your measure?

Do you know when it doesn't work?

Business does not care about rigor, but then terrible things happen when your face recognition system happens to not work for people with dark skin tone.

Even more terrible things happen when it detects cats as people, and worse once it's used to link data into a police database.

Even pretty bad things happen when it's used for detecting potential (treated as absolute) copyright violations.

Security holes when it's used for detecting security problems.

Loss of business when it's used for ticket prioritization.

Business world does not care that it sells a broken solution as long as it's not obvious and someone has been paid. Everyone else pays for the failures. And you cannot sue an ML system really.

Re: What’s the difference between statistics and machine learning?

#75

Post by bumby »

TuringTest wrote: post #64
AstralStorm wrote: post #55

Earlier quoted context omitted.

Formal understanding is critical to actually know the limitations of any given system. Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains.

> Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. On the other hand, it can work indefinitely if it solves a problem well enough and it is used always on the same kind of problem. Sure, you will need to understand it better to apply it to new markets, but often that kind of research is outside the scope of a single business, and they d…

I don't necessarily disagree (in the sense that all models are wrong but some are useful) but to play devils advocate, its damn near impossible to claim a model will work indefinitely on a subset of problems if we don't know it's underpinnings.

Its like saying software has been tested completely when we really should be saying its passed a subset of test cases out of the complete set of possible scenarios.

The only way we can make that claim is if we can interpret the model in a mathematical proof.

Re: What’s the difference between statistics and machine learning?

#76

Post by rockinghigh »

lottin wrote: post #28

Inferential statistics is about explaining an observed outcome in terms of its causing factors. Once we have explained it, then we can make predictions. Machine learning skips the explaining part and goes straight to making predictions, without attempting to understand the underlying process that led to the particular outcome. This would be the main difference, in my opinion.

Causal inference and the ability to explain predictions are also fields of study for machine learning.

Re: What’s the difference between statistics and machine learning?

#77

Post by jimbokun »

edbaskerville wrote: post #17

I'm not sure about machine learning specifically, but I heard somewhere that a data scientist is someone who does statistics, on a Mac, in San Francisco.

And there's this helpful translator:

http://statweb.stanford.edu/~tibs/stat315a/glossary.pdf

Re: What’s the difference between statistics and machine learning?

#78

Post by sweeneyrod »

AstralStorm wrote: post #55
analog31 wrote: post #31

Earlier quoted context omitted.

This seems perfectly fair. And I think that historically there was plenty of use for statistics where people didn't care about the formal understanding, so they were doing crude machine learning before the term became widespread. I've sat through lengthy discussions of machine learning exercises, and could not silence the voice in my head, saying: "This is just curve fitting." Fitting data to an arbitrary curve, and…

Formal understanding is critical to actually know the limitations of any given system. Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains.

> Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains.

That's not my experience at all. What kind of ML requires something beyond basic linear algebra? Being comfortable manipulating matrices is certainly harder than plugging data into an sklearn function, but it's also significantly easier than understanding general relativity.

Re: What’s the difference between statistics and machine learning?

#80

Post by Retric »

sweeneyrod wrote: post #78
AstralStorm wrote: post #55

Earlier quoted context omitted.

Formal understanding is critical to actually know the limitations of any given system. Treating it as a black box has short lifetime, as your method of analysis will miss key features of the system or oversimplify it. Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains.

> Understanding modern ML algebra is really "general relativity hard" if not actually harder. Spiking NNs are "quantum physics hard". The math is very much translatable between these domains. That's not my experience at all. What kind of ML requires something beyond basic linear algebra? Being comfortable manipulating matrices is certainly harder than plugging data into an sklearn function, but it's also significantl…

The NN complexity is not a question of doing the calculations. Go on a 5x5 board would be a very straightforward game, go in a 19x19 board is not. Similarly, understanding how NN fiction at a nuts and bolts level stops being enough as you build larger networks.
Post reply on HN