Why squared error? (2014)
benkuhn.net
Why squared error? (2014)
1–10 of 101 posts
Re: Why squared error? (2014)
#2The Bayesian formulation for the likelihood function would make this squared error explicitly clear.
Re: Why squared error? (2014)
#3Square often corresponds to power in systems.
Re: Why squared error? (2014)
#4To make it positive and to amplify it (as a side-effect).
BTW, "error" is a misleading term - it communicates some fault, at least in the common sense. Distance would be much better term.
So, "squared distance" makes much more sense, because negative distance is nonsense.
Re: Why squared error? (2014)
#5To make it positive and to amplify it (as a side-effect). BTW, "error" is a misleading term - it communicates some fault, at least in the common sense. Distance would be much better term. So, "squared distance" makes much more sense, because negative distance is nonsense.
Well it will only amplify values > 1.
Re: Why squared error? (2014)
#6Interesting discussion. Not sure about the breakdown between ridge regression and LASSO though. The difference is not in the error term but in the regularization term.
Re: Why squared error? (2014)
#7The Bayesian formulation for the likelihood function would make this squared error explicitly clear.
For Gaussian uncertainty. Which still makes it a much more natural assumption than any other I know.
Re: Why squared error? (2014)
#8The Bayesian formulation for the likelihood function would make this squared error explicitly clear.
Bayesian formulations are not necessary, the Gaussian is the maximum entropy distribution for known mean and squared error.
Re: Why squared error? (2014)
#9To make it positive and to amplify it (as a side-effect). BTW, "error" is a misleading term - it communicates some fault, at least in the common sense. Distance would be much better term. So, "squared distance" makes much more sense, because negative distance is nonsense.
Well it will only amplify values > 1.
That's not correct. Even though the magnitudes of the value in isolation shrinks, the relative magnitudes are still amplified which is what matters.
Consider values 1/2 and 1/4: in the original space it's double but in the squared space it becomes 1/4 and 1/16 so the difference is 4x. Also relevantly if you compare eg 0.9 and 1, the gap between them is amplified after squaring.
Re: Why squared error? (2014)
#10To make it positive and to amplify it (as a side-effect). BTW, "error" is a misleading term - it communicates some fault, at least in the common sense. Distance would be much better term. So, "squared distance" makes much more sense, because negative distance is nonsense.
That's a compelling case for why we should not use "distance", because distance cannot be negative, but the error term can.
Just look at bog-standard linear regressions, say Y_i = m X_i + b + ε_i. It makes no sense to call the ε_i terms "distance".