The basic concept of least-squares modelling
1–6 of 6 posts
Re: The basic concept of least-squares modelling
#2When explaining the least-squares, why not drawing the damn squares in the chart then?
Re: The basic concept of least-squares modelling
#3When explaining the least-squares, why not drawing the damn squares in the chart then?
one of my colleagues was lamenting the fact that he made it all the way through pre-university mathematics education without anyone explaining the geometry link between "x-squared", i.e. "x^2", and the area of a square of side-length x.
so this is a pretty reasonable thing to ask for.
Re: The basic concept of least-squares modelling
#4This is a nice interactive example, but it doesn't explain _why_ one would prefer to use a least-squares estimator over any other arbitrary method. For example, one could measure the error in any other L_p norm - e.g. L_1 or L_{\infty} instead of L_2. What's so special about this one?
To answer that, we have the Gauss-Markov theorem [1] -- provided certain assumptions regarding the data are met, least squares is the best linear unbiased estimator.
Note, however, that no-one is forcing us to choose an estimator that is either linear or unbiased. What if by dropping one of those two somewhat arbitrary constraints we could get a better estimator?
One example of this is the James-Stein estimator [2], which is a biased estimator that has the amusing property of dominating (i.e., always performing no worse than, and performing strictly better than in some case) the bog standard least-squares estimate when one is dealing with data in three or more dimensions (and a few more common assumptions).
Crudely speaking, the James-Stein estimator produces estimates that are biased toward the origin, when compared to estimates produced from ordinary least squares. It is also possible to define a James-Stein estimator that biases toward an arbitrary fixed non-origin vector -- these variants also dominate the bog-standard ordinary least squares estimator! Personally I find this result both intriguing and also reasonably offensive (which means my mathematical intuition needs adjustment).
[1] - https://en.wikipedia.org/wiki/Gauss%E2%80%93Markov_theorem
[2] - https://en.wikipedia.org/wiki/James%E2%80%93Stein_estimator
Re: The basic concept of least-squares modelling
#5@shoo: Pretty interesting, Stein. Thx. Taleb advocates abs. deviation, mean abs. dev. and the like.
OP named reason to pick least squares: "easy to calc" IIRC, the rest of the rationale didn't exist back when it was invented and picked....
Re: The basic concept of least-squares modelling
#6When explaining the least-squares, why not drawing the damn squares in the chart then?
Author (and Blabr dev) here. I have no answer--other than it seems like a really good idea, now that you've said it.
The plotting widget I used is based on flot [0]. It looks like flot can draw simple shapes ([1], see "markings"). I'll look into improving the widget--thanks for the idea.