Linear regression by hand • Hacker News Board

Re: Linear regression by hand

#31

Post by thaumaturgy » Sun, Jan 21, 2018, 1:30 AM UTC

barrkel wrote: post #28

soVeryTired wrote: post #21

Earlier quoted context omitted.

Yes, but why ? Right tool for the right job and all that...

Normally the reason you write anything beyond trivial SQL is because you only have a small amount of code to run and lots of data to run it over. Pushing the code to the data is more efficient than pulling the data to the code. The latter might be conceptually cleaner (though it's debatable, relational is a fairly nice programming model and a lot more consistent and well-founded than object orientation, for one), but…

You're basically spot on here. I have a bunch of rows that need to have trend data crunched and updated pretty frequently. Putting this into MySQL cut the server load quite a bit, and it isn't so much business logic that I feel bad about doing it.

Re: Linear regression by hand

#32

Post by mr_toad » Sun, Jan 21, 2018, 1:54 AM UTC

Rainymood wrote: post #8

Can someone explain to me why this has (so many) upvotes? This is like elementary undergraduate econ stats and kind of trivial? There's very little content either, it's literally a reformulation of the formula, no interesting graphs or geometric interpretation. What I expected from a title like "Linear Regression By Hand" was the minimization of some quadratic error function, by hand (i.e. using pencil and paper).

There aren’t that many good articles on basic statistics that are freely available on the net. Textbooks are expensive, and most of the free material is of dubious quality. Wikipedia is particularly terrible on Statistics.

Re: Linear regression by hand

#33

Post by eat_veggies » Sun, Jan 21, 2018, 2:26 AM UTC

pliny wrote: post #11

dankohn1 wrote: post #6

I remember being blown away as an undergrad that least squares (which I had learned first algrebraiclly) had such an obvious geometric meaning: http://www.statisticshowto.com/wp-content/uploads/2014/11/le... You need to square the values so that points that positives and negative differences (between the points and the trend regression line) don't cancel out.

If you only needed to do that, you could just take the abs error (Even the 1-0 loss function, where every point that the regression hyperplane doesn't pass through contributes 1 to the error, fulfills this criterion).

I'm super new to statistics and math, so can you fill me in on why the error is squared rather than absolute valued? Is it because it's easier to take the derivative of, and therefore minimize analytically?

Re: Linear regression by hand

#34

Post by vecter » Sun, Jan 21, 2018, 3:54 AM UTC

xadhominemx wrote: post #24

tw1010 wrote: post #3

It's way more fun to know how to derive least squares than to memorize some formula: https://see.stanford.edu/materials/lsoeldsee263/05-ls.pdf (page 4)

Also useful to understand least squares as a special case of maximum likelihood estimation. MLE I think is very intuitive.

I know a bit of stats, can you explain why MLE is intuitive?

Re: Linear regression by hand

#35

Post by inlineint » Sun, Jan 21, 2018, 3:02 PM UTC

I thought it was about doing scatter plot on graph paper and trying to draw a line with a ruler so that it “almost all points fit”, then empirically measuring the slope and the intercept. I had an impression that it was the way to go in cases when the requirements for accuracy were not strict and calculators was not around.

Re: Linear regression by hand

#36

Post by taeric » Mon, Jan 22, 2018, 3:05 AM UTC

thaumaturgy wrote: post #30

taeric wrote: post #27

Earlier quoted context omitted.

I would guess access to the data is a decent reason to try. No need to pump the data elsewhere, if this is available. I share your doubt that this is worth it, to be clear.

In this case the amount of data and the frequency with which it needs to be updated made handling this in MySQL more practical. I otherwise would've had to have some process querying and updating the records outside of MySQL and it turns out that that puts a whole lot more load on the server than if I just ask MySQL to do it. And besides, what happened to the hacker ethos? "Because I can" should be justification enou…

Is is cool to hear that it works! And just because one doubts something, doesn't mean one shouldn't necessarily try it. Especially if I'm the one doubting. :)