Abusing linear regression to make a point
goodmath.org
Abusing linear regression to make a point
1–10 of 50 posts
Re: Abusing linear regression to make a point
#2I saw that chart on Twitter and thought it was a joke :-/
You don’t need to know anything about maths to see that that is farcical.
Re: Abusing linear regression to make a point
#3Taleb was recently steaming on Twitter about a similar thing done to supposedly show a correlation between physician salary and covid mortality: https://twitter.com/nntaleb/status/1279954325087891464
He follows it with a few examples of spurious regressions from random data: https://twitter.com/nntaleb/status/1280090844113100801
Re: Abusing linear regression to make a point
#4This physically hurts.
Re: Abusing linear regression to make a point
#5Telling every time someone posts a bad regression they never post the R^2.
Re: Abusing linear regression to make a point
#6> I said that if you had reason to believe in a linear relationship, then you could try to find it. That’s the huge catch to linear regression: no matter what data you put in, you’ll always get a “best match” line out.
This challenge generalizes to all model fitting. Incorrectly assuming a distribution is Gaussian is a big one.
Re: Abusing linear regression to make a point
#7Telling every time someone posts a bad regression they never post the R^2.
You might find this article insightful as to the utility of R-squared – "Is R-squared Useless?"
Re: Abusing linear regression to make a point
#8This article does not capture what is actually wrong with the regression.
First, it's not necessarily wrong to fit a linear regression to data that might not be from a linear model, or that you know to be nonlinear. The data could be linear enough in the region of interest for the line to nonetheless be useful, for example. Sure, you need an underlying linear process if you want certain theorems and guarantees to apply. But with any data set, linear or not, regression still gives the best linear approximation to the conditional expectation function.
Second, the following paragraph seems to imply that small correlations are the same as no correlation, and the reason the regression is problematic is that the correlation is small:
> How does that fit look to you? I don’t have access to the original dataset, so I can’t check it, but I’m guessing that the correlation there is somewhere around 0.1 or 0.2 – also known as “no correlation”.
But small correlations, if they actually exist, can sometimes be of great practical relevance. So that's not it either.
The actual problem is that the correlation isn't statistically significant – there isn't enough evidence to conclude that the observed (small) correlation actually exists, as opposed to being the result of random noise in the data. And indeed, as some other comments here point out, you can get similar graphs by fitting lines to randomly simulated fake data.
(If you prefer a Bayesian gloss: the data isn't informative enough to move you off any reasonable prior with most of its mass around zero. Same principle.)
Re: Abusing linear regression to make a point
#9This person has no idea what they're writing about.
Edited because my post was flagged (I'm not sure why). The definition of correlation coefficient is incorrect, which could have been attributed to a typo, except the author goes on to say "The bottom is, essentially, just stripping the signs away.", suggesting the square root of a sum of squared differences would be the same as the sum of differences, were it not for the signs. That's not how norms work.
The whole paragraph on interpreting a correlation coefficient is particularly painful to read: "... if the correlation is perfect – that is, if the dependent variable increases linearly with the independent, then the correlation will be 1. If the dependency variable decreases linearly in opposition to the dependent, then the correlation will be -1. If there’s no relationship, then the correlation will be 0."
For all its good intentions, I feel like this post hurts more than it helps.
Re: Abusing linear regression to make a point
#10> I said that if you had reason to believe in a linear relationship, then you could try to find it. That’s the huge catch to linear regression: no matter what data you put in, you’ll always get a “best match” line out. This challenge generalizes to all model fitting. Incorrectly assuming a distribution is Gaussian is a big one.
The variable of interest may not have a Gaussian distribution, but its expected value and variance generally are. Sure, there's some pathological cases, but the Cauchy distribution doesn't show up that often.