Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I’m Not a Fan of R-Squared

johnmyleswhite.com

Post reply on HN
Print view View original

51–55 of 55 posts

Re: Why I’m Not a Fan of R-Squared

#51

Post by xoranth »

yummyfajitas wrote: post #21
graeham wrote: post #18

Interesting article and I find it current for some problems I'm working on at the moment. I would add a few challanges. The example is a bit a of a strawman - a log(x) function has unique properties that make the Xmax-Xmin vs R^2 work like that. In real data, rarely does a single-variable 'true model' fit as well as the example either. Context is needed as well - depending on the use of the model, a linear or quadrat…

I'm not the author, but I'm a huge fan of robust regression. I make between $500-2000/month off a trading strategy based on such a method. (The method is basically Bayesian linear regression, but using an error model that has a heavier tail than a gaussian.) But a really important thing when using such methods is the lucas critique. When you need to use robust regression you are definitively in a space where all the…

Could you share more details about the robust regression you are using? All resources I could find online on robust regression would either point to Laplace distributed residuals, or some capped loss function.

Re: Why I’m Not a Fan of R-Squared

#52

Post by dllthomas »

tel wrote: post #50
obastani wrote: post #44

Earlier quoted context omitted.

It's not just philosophical -- there are examples of practical situations where there is a true model (especially in more traditional applications of statistics): - Polling for presidential election: The "true model" is the voting preferences of all 300 million Americans. A (uniformly) random sample of N Americans can be used to estimate the true model roughly with standard error 1/sqrt(N). - Particle physics: The "t…

That's reasonably fair, but it's worth noting that even in these situations the true circumstance might be a little more nuanced than the idea of a "true model" suggests. - In polling, it's a bit of an ideal world idea to think that the true voting preferences of all Americans are (1) fixed, (2) consistently measurable, or (3) even relevant given a lot of people won't vote. These are all sources of unknowns and varia…

> (3) even relevant given a lot of people won't vote

This bit seems trivially resolved by interpreting "voting preferences" to include likelihood of voting.

Re: Why I’m Not a Fan of R-Squared

#53

Post by tomrod »

jsprogrammer wrote: post #43
tomrod wrote: post #41

Earlier quoted context omitted.

We can create them! Suppose I take the function y = log(x) and add random white noise. The function log() and the parameters on the random white noise process are the data generating process. We could then fit a model y = \beta X + \epsilon, and then compare the "true" (first) model to our second model. When the natural world generates our data, the idea behind all this is the same: there is a process which generates…

Can you give a non-synthetic, ie. natural, example data generation process? Edit: I don't know if people don't like the grammar, or what? How about this: Can you give a non-synthetic, ie. natural, example of a data generating process?

Sure! Most of the equations you find in your nearest physics or chemistry books are validated experimentally/empirically. The validation comes by better and better approximating the data generating process.

Re: Why I’m Not a Fan of R-Squared

#54

Post by tel »

dllthomas wrote: post #52
tel wrote: post #50

Earlier quoted context omitted.

That's reasonably fair, but it's worth noting that even in these situations the true circumstance might be a little more nuanced than the idea of a "true model" suggests. - In polling, it's a bit of an ideal world idea to think that the true voting preferences of all Americans are (1) fixed, (2) consistently measurable, or (3) even relevant given a lot of people won't vote. These are all sources of unknowns and varia…

> (3) even relevant given a lot of people won't vote This bit seems trivially resolved by interpreting "voting preferences" to include likelihood of voting.

You can definitely model that, too, but it leads you even further away from being able to say that the "true model" is a physical thing of any nature.

Re: Why I’m Not a Fan of R-Squared

#55

Post by dllthomas »

tel wrote: post #54
dllthomas wrote: post #52

Earlier quoted context omitted.

> (3) even relevant given a lot of people won't vote This bit seems trivially resolved by interpreting "voting preferences" to include likelihood of voting.

You can definitely model that, too, but it leads you even further away from being able to say that the "true model" is a physical thing of any nature.

Ah, yeah, fair point - that does move us away from the physical.
Post reply on HN