What’s the difference between statistics and machine learning?

E.g., in regression analysis, the assumptions are (i) there really is a linear mode with the variables to be used; (ii) typically the data will not fit the model exactly and instead there are errors; (iii) the errors are in the sense of probability independent, all have the same distribution which is Gaussian with mean zero. Then estimate the regression coefficients and get, say, a F ratio statistic on the fit and t-tests on the coefficients. For details can see, say, Mood, Graybill, and Boas, or Morrison, or Draper and Smith, or Rao, etc. If you need detailed references, request them here, but these are all classic references going back decades. In that case, maybe some of current machine learning does qualify as statistics.

Here regression was just an example, however, apparently especially close to much of current machine learning. But there is no end of (i) having data, (ii) having some probabilistic assumptions, (iii) manipulating the data, (iv) getting results, and (v) proving some theorems about the probabilistic properties of the results.

Might argue that in machine learning the results of the training data on the model yields some estimates of some probabilistic properties that can be used to make probabilistic statements about the results of the trained model on future real data.

> You made no assumptions about your data, about the distribution of it, any of it. You just followed an algorithm.

You made implicit assumptions that you are now unaware of, which might come and bite you later (eg: using zip codes or names as a proxy for race in credit scoring models, neural networks overfititng to texture and classifying a leopard print couch as a leopard). This means that you are liable to overfit to the data, and generalize poorly, or in ways you are not supposed to.

This is what leads to the perspective that "machine learning" might end up being a powerful tool for laundering bias: https://idlewords.com/talks/sase_panel.htm

So do we need an explicit taxonomy to say that one application is machine learning and another is statistics? Or does the form not matter as much as the function?

Pace the OP, who I'm sure didn't do this with this end in mind, but a lot of the cases I've seen of people trying to bring this up come down to wanting to self-identify in a certain way as opposed to actually talking about the subject at hand.