Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Linear regression by hand

dsgazette.com

Post reply on HN
Print view View original

11–20 of 36 posts

Re: Linear regression by hand

#11

Post by pliny »

dankohn1 wrote: post #6

I remember being blown away as an undergrad that least squares (which I had learned first algrebraiclly) had such an obvious geometric meaning: http://www.statisticshowto.com/wp-content/uploads/2014/11/le... You need to square the values so that points that positives and negative differences (between the points and the trend regression line) don't cancel out.

If you only needed to do that, you could just take the abs error (Even the 1-0 loss function, where every point that the regression hyperplane doesn't pass through contributes 1 to the error, fulfills this criterion).

Re: Linear regression by hand

#12

Post by VHRanger »

Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

I think a lot of people here didn't take econometrics courses.

If you see all regression problems under the foil of maximum likelihood estimation, you might not know that ordinary least squares regression has a closed form solution

Re: Linear regression by hand

#13

Post by bluusteel »

Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

It seems like blog posts on simple statistical methods like this one land on the front page of HNews a lot more than one would expect.

Re: Linear regression by hand

#14

Post by VHRanger »

theophrastus wrote: post #10
Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

One issue that I nearly always find missing in intro discussions about linear regression is the near universal assumption of no error in the abcissal/"x" values. And while this is true-ish for time series data, (we know for certain which day we collected the data on - yet the same hour every day?), I'd be rich if I had a nickel for every time I saw standard linear regression done when the "x" had significant (and kno…

Regression with measurement error is usually treated in much higher level statistics/econometrics classes.

If you're interested in this you can read more in Mostly Harmless Econometrics [1] about adressing this with IV methods

[1] http://www.development.wne.uw.edu.pl/uploads/Main/recrut_eco...

Re: Linear regression by hand

#16

Post by pacbard »

VHRanger wrote: post #14
theophrastus wrote: post #10

Earlier quoted context omitted.

One issue that I nearly always find missing in intro discussions about linear regression is the near universal assumption of no error in the abcissal/"x" values. And while this is true-ish for time series data, (we know for certain which day we collected the data on - yet the same hour every day?), I'd be rich if I had a nickel for every time I saw standard linear regression done when the "x" had significant (and kno…

Regression with measurement error is usually treated in much higher level statistics/econometrics classes. If you're interested in this you can read more in Mostly Harmless Econometrics [1] about adressing this with IV methods [1] http://www.development.wne.uw.edu.pl/uploads/Main/recrut_eco...

To build on this a little bit more, there are also generalized linear models that allow to specify the reliability (i.e., error level) of a variable.

Regarding, 2SLS models, I find them more useful to account for endogeneity in the model rather than measurement error. After all, measurement error is usually unobserved (otherwise you would just take it out). 2SLS “just” reweigh the point estimates by identifying the good variation in the instrumented variable using the instrument (for example, using draft lottery results to account for the endogenous choice to attend college).

Re: Linear regression by hand

#17

Post by thess24 »

Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

I agree, it seems like no help just to list a formula to memorize. If someone knows enough linear algebra to understand what the formula represents, they can do the derivation. This link [1] is a good one if anyone is interested.

[1] https://eli.thegreenplace.net/2014/derivation-of-the-normal-...

Re: Linear regression by hand

#18

Post by luckyt »

Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

Well, unless you're doing original research, everything you write will be trivial to somebody.

Re: Linear regression by hand

#20

Post by thaumaturgy »

Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

https://www.xkcd.com/1053/

I almost certainly know something I'd consider "trivial" that you haven't encountered yet. I try to be really excited when that happens.

Post reply on HN