You would probably get what you want with a Deming regression.
Why does a least squares fit appear to have a bias when applied to simple data?
11–20 of 84 posts
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#12Earlier quoted context omitted.
I find it helpful to view least as fitting the noise to a Gaussian distribution.
OLS estimator is the minimum-variance linear unbiased estimator even without the assumption of Gaussian distribution.
Yes, and if I remember correctly, you get the Gaussian because it's the minimum entropy (least additional assumptions about the shape) continuous distribution given a certain variance.
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#13Linear Regression a.k.a. Ordinary Least Squares assumes only Y has noise, and X is correct.
Your "visual inspection" assumes both X and Y have noise. That's called Total Least Squares.
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#14My head canon:
If the true value is medium high, any random measurements that lie even further above are easily explained, as that is a low ratio of divergence. If the true value is medium high, any random measurements that lie below by a lot are harder to explain, since their (relative, i.e.) ratio of divergence is high.
Therefore, the further you go right in the graph, the more a slightly lower guess is a good fit, even if many values then lie above it.
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#15Had a QuantSci Prof who was fond of asking "Who can name a data collection scenario where the x data has no error?" and then taught Deming regression as a generally preferred analysis [1]
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#16Sorry for my negativity / meta comment on this thread. From what I can tell the stackexchange discussion in the submission already to provides all the relevant points to be discussed about this.
While the asymmetry of least squares will probably be a bit of a novelty/surprise to some, pretty much anything posted here is more or less a copy of one of the comments on stackexchange.
[Challenge: provide a genuinely novel on-topic take on the subject.]
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#17Many times I've looked at the output of a regression model, seen this effect, and then thought my model must be very bad. But then remember the points made elsewhere in thread.
One way to visually check that the fit line has the right slope is to (1) pick some x value, and then (2) ensure that the noise on top of the fit is roughly balanced on either side. I.e., that the result does look like y = prediction(x) + epsilon, with epsilon some symmetric noise.
One other point is that if you try to simulate some data as, say
y = 1.5 * x + random noise
then do a least squares fit, you will recover the 1.5 slope, and still it may look visually off to you.
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#18A note mostly about terminology:
The least squares model will produce unbiassed predictions of y given x, i.e. predictions for which the average error is zero. This is the usual technical definition of unbiassed in statistics, but may not correspond to common usage.
Whether x is a noisy measurement or not is sort of irrelevant to this -- you make the prediction with the information you have.
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#19[deleted]
Re: Why does a least squares fit appear to have a bias when applied to simple data?
#20Had a QuantSci Prof who was fond of asking "Who can name a data collection scenario where the x data has no error?" and then taught Deming regression as a generally preferred analysis [1] [1] https://en.wikipedia.org/wiki/Deming_regression
From that wikipedia article, delta is the ratio of y variance to x variance. If x variance is tiny compared to y variance (often the case in practice) then will we not get an ill-conditioned model due to the large delta?