(2022)
Everything is a linear model
31–40 of 90 posts
Re: Everything is a linear model
#32RSS feed: https://danielroelfs.com/blog/index.xml
Re: Everything is a linear model
#33Earlier quoted context omitted.
"Classification of mathematical problems as linear and nonlinear is like classification of the Universe as bananas and non-bananas. " and everything turns around the same principles. For example dynamical models and PID controls. yet solving a banana, is the only thing we really know how to do. So we end up fitting everything in our banana models.
I disagree with the implication that linearity is an unnatural concept, it appears whenever the changes being studied are small relative to the key parameters that determine the system. Every system is linear for small perturbations. Even logic gates; in negative feedback they can form passable inverting amplifiers. In a place as big as the universe it is rather common for two things to be very different in scale and…
> Every system is linear for small perturbations.
every smooth system, sure, but even continuity is no guarantee of locally linearity.
Re: Everything is a linear model
#34A cool thing about linear models is that they can be used to model non-linear correlations by using transformations. For example, an exponential function can be made linear if you just take the logarithm.
Re: Everything is a linear model
#35Statistics is more than hypothesis testing, but you'll get surprisingly far without straying too far from linear models - I remember a Stats prof saying 'most of classical Statistics is GLM [0]' [0] https://en.wikipedia.org/wiki/Generalized_linear_model
"Classification of mathematical problems as linear and nonlinear is like classification of the Universe as bananas and non-bananas. " and everything turns around the same principles. For example dynamical models and PID controls. yet solving a banana, is the only thing we really know how to do. So we end up fitting everything in our banana models.
My favorite moment in university was in the first class of semester 2, where a prof said "lets look at a really small part of our thing, and assume we apply some force to it. This will make it stretch, lets assume the stretching is linear relative to the applied force". I raised my hand and asked "is this assumption supported empirically?" and he said "no, we know it's not always true, but if we don't make it we can't calculate anything".
At the time I was mad at engineers for being non-scientific, but after a few years I understood the deep wisdom in that. Nonlinear materials exist, and materials we use have nonlinear ranges. We just don't build things from those, because the math is too unwieldy. (except in very very specific edge cases where we spend a lot of money building a very limited thing)
Re: Everything is a linear model
#36I thought nonlinearity was very important to be able to make a larger model better than a smaller one? Like so important that tom7 made a half-joke demo with it: https://yewtu.be/watch?v=Ae9EKCyI1xU
Well, you can create a non-linear model by piece-wise combining multiple linear models. The famous ReLU non-linearity is just that - two linear functions joined.
I see what you mean. Though in my mind, and this is clearly subjective, piece-wise linear is at least less strict than a linear model. (With enough ReLUs, you could get arbitrarily close to a lookup table, which I think would be best described as a nonlinear model.)
Re: Everything is a linear model
#37You can state and prove theorems with linear models. You can do inference and testing. This means papers and academics naturally love them. And therefore, they are everywhere.
Not the case with non-linear models. We need to throw computers at them.
Re: Everything is a linear model
#38A cool thing about linear models is that they can be used to model non-linear correlations by using transformations. For example, an exponential function can be made linear if you just take the logarithm.
See the topic of GLMs: https://en.wikipedia.org/wiki/Generalized_linear_model
Re: Everything is a linear model
#39A cool thing about linear models is that they can be used to model non-linear correlations by using transformations. For example, an exponential function can be made linear if you just take the logarithm.
Not all functions have a trivial transformation.
Re: Everything is a linear model
#40Earlier quoted context omitted.
I disagree with the implication that linearity is an unnatural concept, it appears whenever the changes being studied are small relative to the key parameters that determine the system. Every system is linear for small perturbations. Even logic gates; in negative feedback they can form passable inverting amplifiers. In a place as big as the universe it is rather common for two things to be very different in scale and…
> Every system is linear for small perturbations. every smooth system, sure, but even continuity is no guarantee of locally linearity.
I've never actually seen a physical example of a system without a continuous first derivative. For example phase transitions, commonly touted as an example of discontinuity, don't actually occur until the matter has gone a bit over the point and a transition nucleates somewhere. The probability of a phase transition is a continuous function of temperature, with continuous derivatives.
I'm skeptical that discontinuities can exist because, if they did, they'd serve as infinitely powerful microscopes. If there's a discontinuity in nature, it must exist at absolute zero. I don't have a similarly good argument for continuous first derivatives but I do think it's interesting that there are no examples AFAIK.