Linear regression by hand
dsgazette.com
Linear regression by hand
1–10 of 36 posts
Re: Linear regression by hand
#2[deleted]
Re: Linear regression by hand
#3It's way more fun to know how to derive least squares than to memorize some formula: https://see.stanford.edu/materials/lsoeldsee263/05-ls.pdf (page 4)
Re: Linear regression by hand
#4It's way more fun to know how to derive least squares than to memorize some formula: https://see.stanford.edu/materials/lsoeldsee263/05-ls.pdf (page 4)
[misunderstood]
Re: Linear regression by hand
#5It's way more fun to know how to derive least squares than to memorize some formula: https://see.stanford.edu/materials/lsoeldsee263/05-ls.pdf (page 4)
[misunderstood]
You're confusing linear regression and least squares. (They're connected but not identical in that way.) Least squares gives the closest orthagonal (perpendicular) projection onto the range of the matrix. The slide is correct.
Re: Linear regression by hand
#6I remember being blown away as an undergrad that least squares (which I had learned first algrebraiclly) had such an obvious geometric meaning:
http://www.statisticshowto.com/wp-content/uploads/2014/11/le...
You need to square the values so that points that positives and negative differences (between the points and the trend regression line) don't cancel out.
Re: Linear regression by hand
#7MySQL is perfectly capable of calculating a linear regression for you, btw. In my case, I needed to be able to estimate trends from sparse time series data. Here's how you do that:
SELECT
@a_count := avg(count) as mean_count,
@a_weeks := avg(`week`) as mean_weeks,
@covariance := (sum(`week` * `count`) - sum(`week`) * sum(`count`) / count(`week`)) / count(`week`) as covariance,
@stddev_count := stddev(`count`) as stddev_count,
@stddev_week := stddev(`week`) as stddev_week,
@r := @covariance / (@stddev_count * @stddev_week) as r,
@slope := @r * @stddev_count / @stddev_week as slope,
@y_int := @a_count - (@slope * @a_weeks) as y_int,
@this_week_no := timestampdiff(WEEK, (select min(`date`) from dataset), curdate()) as this_week_no,
@predicted := round(greatest(1, @y_int + (@slope * @this_week_no))) as predicted
FROM (SELECT timestampdiff(WEEK, (select min(`date`) from dataset), `date`) as week, count(date) as count FROM dataset group by WEEK(date)) series;
This performs well enough to be able to crunch tens of millions of rows of data in "reasonable time" on a wimpy VPS.
Re: Linear regression by hand
#8Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial?
There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).
Re: Linear regression by hand
#9Very small fully connected neural networks are incredibly good at approximating functions even after one second of training with RPROP. Of course for complex non linear functions as well.
Btw doing linear regression with pencil and paper just geometrically tracing a line that appears to fit the points and then calculating then coefficients is trivial.
Re: Linear regression by hand
#10Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).
One issue that I nearly always find missing in intro discussions about linear regression is the near universal assumption of no error in the abcissal/"x" values. And while this is true-ish for time series data, (we know for certain which day we collected the data on - yet the same hour every day?), I'd be rich if I had a nickel for every time I saw standard linear regression done when the "x" had significant (and known) error. In which case you're biasing yourself unless you use some sort of 2d regression, like Deming.[1]