The entire product I built over the last year can be reduced to basic statistics (e.g. ratios, probabilities) but because of the hype train we build "models" and "predict" certain outcomes over a data set. One of the products the company I work for sells more or less attempts to find duplicate entries in a large, unclean data set with "machine learning." The value added isn't in the use of ML techniques itself, it's…
It's interesting to me that with all the ML hype, it's still not clear what constitutes ML. A basic k-means or naive Bayes approach will show up in ML textbooks, but those aren't clearly different from "use some statistics to make a prediction". There's an interesting group of marginal approaches that have existed as-is for years, but have increasingly focused their branding on machine learning as its profile has ris…
In brief, you're going to run up against two types of data - categorical and continuous. (There are facets to this, eg ordinal, but these are really the elemental types of data). The relationship of datatype to independent/dependent variable is what determines what kind of analysis you may conduct.
Categorical Independent vs. Categorical Dependent, for example, is fairly restrictive, as makes logical sense. You may cross-tabulate, you may score likelihood based on previous observation, but obviously, because all of the data involved are non-numeric, there's no chance for regression, ANOVA, etc. Linear Regression is used when both independent and dependent variables are continuous, and cross-category differencing techniques like ANOVA may be used when the independent is categorical and the dependent is continuous.
The part you don't typically learn until grad school is when the independent is continuous and the dependent is categorical, ie, in ML, a classification problem. The standard statistical methods used as foundation for these problems are logistic regression, logit/probit. It's expansion of these methods that lead to ML in the first place.