Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Everything is a linear model

danielroelfs.com

Post reply on HN
Print view View original

71–80 of 90 posts

Re: Everything is a linear model

#71

Post by new_username01 »

idiotsecant wrote: post #61
levocardia wrote: post #53

I see a lot of comments here assuming "linear model" means "can't model nonlinearities." Absolutely not the case. Splines can easily take care of that. The "linear" part of linear model just means "linear in the predictor space." You can add a non-linear predictor easily via spline basis (similar/sometimes identical to "kernels" in ML). My series of lm/glm/gam/gamm revelations was: 1. All t-tests and ANOVA flavors ar…

Widely known in the controls engineering world for the last hundred years as 'anything is linear if you zoom in far enough'

It's amazing how many ideas I've seen being "rediscovered" these days from control theory.

Re: Everything is a linear model

#72

Post by 2-718-281-828 »

levocardia wrote: post #53

I see a lot of comments here assuming "linear model" means "can't model nonlinearities." Absolutely not the case. Splines can easily take care of that. The "linear" part of linear model just means "linear in the predictor space." You can add a non-linear predictor easily via spline basis (similar/sometimes identical to "kernels" in ML). My series of lm/glm/gam/gamm revelations was: 1. All t-tests and ANOVA flavors ar…

> Splines can easily take care of that.

within set boundaries.

Re: Everything is a linear model

#74

Post by usgroup »

levocardia wrote: post #53

I see a lot of comments here assuming "linear model" means "can't model nonlinearities." Absolutely not the case. Splines can easily take care of that. The "linear" part of linear model just means "linear in the predictor space." You can add a non-linear predictor easily via spline basis (similar/sometimes identical to "kernels" in ML). My series of lm/glm/gam/gamm revelations was: 1. All t-tests and ANOVA flavors ar…

GLMs are a non linear transformation on an output followed by linear modelling. They are referred to as “linear models” but you might as well then also consider NNs as linear models, or any model at all which ends with addition as the final step.

Re: Everything is a linear model

#75

Post by EvgeniyZh »

ivansavz wrote: post #70
EvgeniyZh wrote: post #69

Earlier quoted context omitted.

"You can make linear model non linear by adding non linearity" is kind of vacuous statement, isn't it

No no, linear just means some expression of the form y = b1*x1 + b2*x2 + ... + bp*xp which is in the heart of the model (perhaps more specific description would be linear combination). Whatever you call it, it's just some quantity y, that is constructed via an additive process from components x1...xp, and each component is multiplied by some constant (the coefficient of the linear combination). This linear combinatio…

Well yeah, but what is the non trivial statement here? Linear model is linear, and if you add to your model non-linear steps (such as pre- or post- processing) then it is not longer linear and can handle non-linear data. In the same spirit, you could say that GPT4 is a linear model (token logits) with a very large preprocessing (transformer).

Maybe what OP meant is that particular non linearities (splines) allow to keep some goodies from the linear model in non-linear settings. Still it's not clear for me in which settings exactly such models are good enough

Re: Everything is a linear model

#76

Post by kqr »

idiotsecant wrote: post #61
levocardia wrote: post #53

I see a lot of comments here assuming "linear model" means "can't model nonlinearities." Absolutely not the case. Splines can easily take care of that. The "linear" part of linear model just means "linear in the predictor space." You can add a non-linear predictor easily via spline basis (similar/sometimes identical to "kernels" in ML). My series of lm/glm/gam/gamm revelations was: 1. All t-tests and ANOVA flavors ar…

Widely known in the controls engineering world for the last hundred years as 'anything is linear if you zoom in far enough'

Wait isn't that just an application of Taylor expansion which analysis has known about for several hundreds of years?

Re: Everything is a linear model

#77

Post by kqr »

This is why I feel like I couldn't become a credible AI/ML consultant. I would just throw everything into a linear model, make some progress, and call it a day.

Re: Everything is a linear model

#78

Post by f1shy »

kqr wrote: post #76
idiotsecant wrote: post #61

Earlier quoted context omitted.

Widely known in the controls engineering world for the last hundred years as 'anything is linear if you zoom in far enough'

Wait isn't that just an application of Taylor expansion which analysis has known about for several hundreds of years?

Taylor is only for some functions, must be continuos and differentiable to n grades. In control theory you break the functions, so discontinuities can also be considered.

Re: Everything is a linear model

#80

Post by blitzar »

kqr wrote: post #77

This is why I feel like I couldn't become a credible AI/ML consultant. I would just throw everything into a linear model, make some progress, and call it a day.

99% of AI / ML consultants are just throwing everything into a linear model, make no progress, slap a big AI sticker on the box and call it a day.

You would have fitted right in.

Post reply on HN