Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Types of Regression Analysis

listendata.com

Post reply on HN
Print view View original

11–20 of 76 posts

Re: Types of Regression Analysis

#11

Post by matchagaucho »

swirepe wrote: post #4
matchagaucho wrote: post #3

Are there any ML APIs or web services that accept a vector and run various regression scenarios to identify optimal fit? I suppose vectors for both training and testing would be required. Would gladly pay $1-$5 per batch for a service to do this.

Why wouldn't you just run weka or something locally?

I just want to provide the data and let a service decide the best algorithm.

Weka and various ML tools require you select the algorithm and do the A/B testing on your own.

There's an opportunity for an Optimizely of ML.

Re: Types of Regression Analysis

#12

Post by gmfawcett »

nerdponx wrote: post #9
gmfawcett wrote: post #6

> Assumptions of linear regression: There must be a linear relation between independent and dependent variables. That's not wrong, but it's a strong way to word it. If linear regression were only suitable when the variables were perfectly linearly related, it would get a lot less use. Practically, linear regression can be used when the relationship is linear-ish, at least in the interval of interest. In other words,…

That the linear model is "correct" is only assumption if you're trying to draw probabilistic inferences. There's nothing stopping you from using it as a "best fit line", even when you have no reason to believe those assumptions. But then it's just a best-fit line. It tells you the direction and magnitude of linear trend, nothing more. That's never wrong in any sense, it's just that sometimes it's not very useful.

Yes, well put!

Re: Types of Regression Analysis

#13

Post by snicker7 »

Despite what the article claims, normality is not actually an assumption of linear regression. It is "required" for doing F-tests (the F-distribution being related to the normal distribution), but it is not required for proving that the regression coefficients are consistent.

Re: Types of Regression Analysis

#14

Post by meanmrmustard92 »

gmfawcett wrote: post #6

> Assumptions of linear regression: There must be a linear relation between independent and dependent variables. That's not wrong, but it's a strong way to word it. If linear regression were only suitable when the variables were perfectly linearly related, it would get a lot less use. Practically, linear regression can be used when the relationship is linear-ish, at least in the interval of interest. In other words,…

It is actually wrong. The assumption is that y is a linear combination of the covariates in X. You can run regressions like y = x + x^2 (i.e. you permit a quadratic relationship) just fine.

Re: Types of Regression Analysis

#16

Post by thanatropism »

This is just horrible quality material. What in the heck is this?

> It is to be kept in mind that the coefficients which we get in quantile regression for a particular quantile should differ significantly from those we obtain from linear regression. If it is not so then our usage of quantile regression isn't justifiable. This can be done by observing the confidence intervals of regression coefficients of the estimates obtained from both the regressions.

Re: Types of Regression Analysis

#17

Post by bunderbunder »

meanmrmustard92 wrote: post #14
gmfawcett wrote: post #6

> Assumptions of linear regression: There must be a linear relation between independent and dependent variables. That's not wrong, but it's a strong way to word it. If linear regression were only suitable when the variables were perfectly linearly related, it would get a lot less use. Practically, linear regression can be used when the relationship is linear-ish, at least in the interval of interest. In other words,…

It is actually wrong. The assumption is that y is a linear combination of the covariates in X. You can run regressions like y = x + x^2 (i.e. you permit a quadratic relationship) just fine.

It's not wrong, it's just a way of looking at things that speaks to the underlying math rather than the full extent of what you can do with it if you extend it with things like kernel methods.

When you use linear regression to fit a model like

  y ~ ax + b(x^2)
what you're technically doing is fitting a linear function with two parameters on two variables. One variable happens to always be equal to the square of the other variable, but, for the purpose of how the model is usually going to be fit, it is still using the same old analytical method that's based in linear algebra.

Re: Types of Regression Analysis

#18

Post by [deleted] »

snicker7 wrote: post #13

Despite what the article claims, normality is not actually an assumption of linear regression. It is "required" for doing F-tests (the F-distribution being related to the normal distribution), but it is not required for proving that the regression coefficients are consistent.

[deleted]

Re: Types of Regression Analysis

#19

Post by knoepfle »

snicker7 wrote: post #13

Despite what the article claims, normality is not actually an assumption of linear regression. It is "required" for doing F-tests (the F-distribution being related to the normal distribution), but it is not required for proving that the regression coefficients are consistent.

It's actually not even required for that! See http://davegiles.blogspot.com/2011/08/being-normal-is-option... which cites King (1980):

> If the error vector in our regression model follows any distribution in the family of Elliptically Symmetric distributions, then any test statistic that is scale-invariant has the same null and alternative distributions as they have when the errors are normally distributed.

Re: Types of Regression Analysis

#20

Post by thanatropism »

matchagaucho wrote: post #3

Are there any ML APIs or web services that accept a vector and run various regression scenarios to identify optimal fit? I suppose vectors for both training and testing would be required. Would gladly pay $1-$5 per batch for a service to do this.

I have a magic regression aggregator that works like this:

1) Take a dataset and split into training and test

2) Using the training set: run a bunch of different regressors (for a training-training subset) and get predictions (for the remaining test-training subset)

3) Run a higher-level regression against test-training subset predictions. I use either plain linear regression (so my meta-regressor is a linear combination of the regressors) or K-nearest neighbors (so the best regressor for each region of feature space is chosen).

4) If there are hyperparameters, optimize against the test set (not the test-training subset).

It's not available as an API. I'm available for consulting though.

Post reply on HN