Another fun stats X is really Y:
estimating the Area Under the Curve metric (AUC) is equivalent to the Wilcoxon-Mann-Whitney test!
https://rmets.onlinelibrary.wiley.com/doi/abs/10.1256/003590...
Everything is a linear model
11–20 of 90 posts
Re: Everything is a linear model
#12Hmm...I thought everything was an Eigenfunction.
Applying basic syllogism, Eigenfunction is securities fraud.
Re: Everything is a linear model
#13Statistics is more than hypothesis testing, but you'll get surprisingly far without straying too far from linear models - I remember a Stats prof saying 'most of classical Statistics is GLM [0]'
Re: Everything is a linear model
#14Earlier quoted context omitted.
Linear models don’t need everything to be linear.
I presume you are implying that linear model only mandates linear relationship between predictor and regression coefficients?
A linear relationship between any transformation of the outcome and any transformation of the predictor variables — so the function is linear but the relationship between predictors and outcome can take on almost any shape.
Re: Everything is a linear model
#15Earlier quoted context omitted.
I presume you are implying that linear model only mandates linear relationship between predictor and regression coefficients?
A linear relationship between any transformation of the outcome and any transformation of the predictor variables — so the function is linear but the relationship between predictors and outcome can take on almost any shape.
Ah, I missed 'the transformation of outcome' in my mind. Thanks for clearing it up.
Re: Everything is a linear model
#16Earlier quoted context omitted.
Linear models don’t need everything to be linear.
I presume you are implying that linear model only mandates linear relationship between predictor and regression coefficients?
Linear models are a linear combination of possibly non-linear regressors. The linearity is strictly in the parameters, not in whatever you're adding up.
A neural network can be pedantically referred to as a linear model of the form y = a + b*neural_network, for example. Here, y is a linear model (even though neural_network isn't).
Re: Everything is a linear model
#17I thought nonlinearity was very important to be able to make a larger model better than a smaller one? Like so important that tom7 made a half-joke demo with it: https://yewtu.be/watch?v=Ae9EKCyI1xU
Nonlinear things start looking like linear things again in very high dimensions
Re: Everything is a linear model
#18Statistics is more than hypothesis testing, but you'll get surprisingly far without straying too far from linear models - I remember a Stats prof saying 'most of classical Statistics is GLM [0]' [0] https://en.wikipedia.org/wiki/Generalized_linear_model
Which means it really all just finding hyperplanes that are near the data.
Re: Everything is a linear model
#19I thought everything was a power function.
Re: Everything is a linear model
#20I thought nonlinearity was very important to be able to make a larger model better than a smaller one? Like so important that tom7 made a half-joke demo with it: https://yewtu.be/watch?v=Ae9EKCyI1xU
Well, you can create a non-linear model by piece-wise combining multiple linear models. The famous ReLU non-linearity is just that - two linear functions joined.
same thing with any feed forward network too. They are all piece-wise linear in respect to inputs.
Layers reduce resource requirements and make some patterns easier or even practical to find, but any ANN that is a FNN supervised learning could be represented as a parametric linear regression.
Unsupervised learning, that tends to use clustering is harder to visualize but is the same thing.
You still have ANNs, which have binary output, which can be viewed through the lens of deciders. They have to have unique successor and predecessor functions.
Really this is just set shattering that relates to a finite VC dimensionality being required for something to be PAC learnable.
But the title of this is confusing the map for the territory. It isn't that 'Everything is a linear model' but that linear models are the preferred, most practical form.
The efforts to leverage spikey neutral networks, which is a more realistic model of cortical neurons, and which have continuous output (or more correctly the computable reals) tend to run into problems like riddled basins.
https://arxiv.org/abs/1711.02160
Obviously setting rectified linear unit at 0 = 1 resolves to differentiation problem, but many functions may not be so simple
Perhaps a useful lens is how TSP with a discreet Euclidean metric is in NP-complete while the continuous version is in NP-hard.
But it isn't that everything is linearizable, but rather that linearized problems tend to be the most practical.