Heyyy man, what if the universe is LINEAR?
"Classification of mathematical problems as linear and nonlinear is like classification of the Universe as bananas and non-bananas."
I realized that derivatives are linear
21–30 of 76 posts
Re: I realized that derivatives are linear
#22This is why you take linear algebra and calculus before doing machine learning.
The dude/dudette figured it out on their own which is what math is actually about. Who cares if it's well known? Knowing a fact because you learned it in a class is cool and all but it's not the only way to acquire knowledge, and I would argue not even the best way---just the most efficient fact/hour ratio.
Re: I realized that derivatives are linear
#23Well this is the whole point of derivatives (i.e. tangent maps): to be linear approximations of functions. So yes, a linear approximation of a linear function is the function itself.
If a function f passes through some point (a,b), then the tangent to f through that point is given by (y-b) = f'(a)·(x-a) and that function is affine but usually not linear. (For the tangent curve to be a linear function, you would need a·f'(a) = b, so that the tangent goes through the point (0,0).) It's not at all obvious to me that this means that the function d(f) = df/dx is linear. It is linear, but I don't see h…
It is common in this context to say "linear" to also mean "affine", because after all affine functions are not much more complicated than linear functions.
Re: I realized that derivatives are linear
#24Well this is the whole point of derivatives (i.e. tangent maps): to be linear approximations of functions. So yes, a linear approximation of a linear function is the function itself.
That is my feeling after reading this article. For people who have learned a little bit of Analysis, it is like saying "the earth is round". There is nothing new. The Wikipedia page of Derivative (https://en.wikipedia.org/wiki/Derivative) has a detailed description on the linearity. (Well, I do appreciate the author's way of presenting the idea, but I don't think it deserves an in-depth discussion on Hacker News.)
Re: I realized that derivatives are linear
#25Earlier quoted context omitted.
You are right, but what you are saying has nothing to do with the author's point: what he is saying is that the differentiation operator itself is linear, which is a meaningful and true fact even in spaces where you have no idea of what a linear function is.
I think you'll find that derivatives are overwhelming defined as operators on vector spaces. It is definitely correct to talk about a linear function on a vector space.
I agree with you, but this is not what GGP is saying. GGP is saying another true fact (i.e., that derivative of a linear function is that same function), which a different thing than stating, as the article says, that the differentiation operator is linear. On a manifold there is no concept of a linear function, so you cannot say that the derivative of a linear function is the same function, but the differentiation operator is still defined and linear.
Re: I realized that derivatives are linear
#26There's a ² missing. Should be dC/dx = sum d/dx |...|².
Re: I realized that derivatives are linear
#27Earlier quoted context omitted.
I think you'll find that derivatives are overwhelming defined as operators on vector spaces. It is definitely correct to talk about a linear function on a vector space.
I agree with you, but this is not what GGP is saying. GGP is saying another true fact (i.e., that derivative of a linear function is that same function), which a different thing than stating, as the article says, that the differentiation operator is linear. On a manifold there is no concept of a linear function, so you cannot say that the derivative of a linear function is the same function, but the differentiation o…
Yes, you are right. My apologies.
Re: I realized that derivatives are linear
#28Earlier quoted context omitted.
I think you'll find that derivatives are overwhelming defined as operators on vector spaces. It is definitely correct to talk about a linear function on a vector space.
I agree with you, but this is not what GGP is saying. GGP is saying another true fact (i.e., that derivative of a linear function is that same function), which a different thing than stating, as the article says, that the differentiation operator is linear. On a manifold there is no concept of a linear function, so you cannot say that the derivative of a linear function is the same function, but the differentiation o…
Of course you are correct, even though one could argue the linearity of differentiation is a property you can obtain by differentiating in coordinate charts, where the reasoning is still valid.
In any case, it is probably a good thing to get a good intuition of what differentiation and derivatives are in the vector space setting before digging into differential geometry.
Re: I realized that derivatives are linear
#29This is why you take linear algebra and calculus before doing machine learning.
The dude/dudette figured it out on their own which is what math is actually about. Who cares if it's well known? Knowing a fact because you learned it in a class is cool and all but it's not the only way to acquire knowledge, and I would argue not even the best way---just the most efficient fact/hour ratio.
You are right. Sometimes the most sticking facts in my head are those that I rediscovered by myself, because that helped me to see their actual importance, which might have not been that apparent while hearing about them in classes. There are teaching theories in which the student is not supposed to learn about something, but they are guided to rediscover it by their tutors.
Re: I realized that derivatives are linear
#30Well this is the whole point of derivatives (i.e. tangent maps): to be linear approximations of functions. So yes, a linear approximation of a linear function is the function itself.
If a function f passes through some point (a,b), then the tangent to f through that point is given by (y-b) = f'(a)·(x-a) and that function is affine but usually not linear. (For the tangent curve to be a linear function, you would need a·f'(a) = b, so that the tangent goes through the point (0,0).) It's not at all obvious to me that this means that the function d(f) = df/dx is linear. It is linear, but I don't see h…
The sum operator is linear, so its derivative is itself: the derivative of the sum is the sum of derivatives. Same goes for multiplication by a scalar.
From this you obtain that the derivative of any linear combination is the linear combination of the derivatives: differentiation is linear.