Better later than never, I guess.
I’ve heard that the next big thing will be this “chain rule”, you should check that out too.
I realized that derivatives are linear
11–20 of 76 posts
Re: I realized that derivatives are linear
#12This is why you take linear algebra and calculus before doing machine learning.
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html
Hm, I feel like how that comment is read is in the eye of the beholder... it just depends on whether you see the glass half empty or half full. You could read it as a dismissal, or you could read it as an advice or a warning that could very well teach someone (like the OP) something... in this case, I suppose that might be something like: "Here's the important thing to really take away from this incident: it's worth taking the courses in the recommended manner so you can learn more insightful things like this with less effort."
Re: I realized that derivatives are linear
#13There seems to be a misconception that linear transformations have to look like lines.
Re: I realized that derivatives are linear
#14Heyyy man, what if the universe is LINEAR?
Re: I realized that derivatives are linear
#15Well this is the whole point of derivatives (i.e. tangent maps): to be linear approximations of functions. So yes, a linear approximation of a linear function is the function itself.
If a function f passes through some point (a,b), then the tangent to f through that point is given by
(y-b) = f'(a)·(x-a)
It's not at all obvious to me that this means that the function d(f) = df/dx is linear. It is linear, but I don't see how the tangent curve demonstrates it.
Re: I realized that derivatives are linear
#16The derivative is a linear operator, but it's not a bounded operator. That is, for example, the vector norm of f(x) = k·sin(x/k) → 0 when k→0, but the norm of d/dx f(x) does not. This also means that it's not continuous. Of the mappings between vector spaces, the most well behaving are the bounded linear operators, and the derivative doesn't belong to these. But yes, it's linear. Edit: Originally wrote f(x) = k·sin(k…
You cannot speak of boundedness of operators if you don't specify the domain and codomain spaces. The derivation operator is for example bounded between C^1(R) and C^0(R), or between C^\infty(R) and C^\infty(R). Your example correctly proves that it is not bounded between C^0(R) and C^0(R).
Re: I realized that derivatives are linear
#17The derivative is a linear operator, but it's not a bounded operator. That is, for example, the vector norm of f(x) = k·sin(x/k) → 0 when k→0, but the norm of d/dx f(x) does not. This also means that it's not continuous. Of the mappings between vector spaces, the most well behaving are the bounded linear operators, and the derivative doesn't belong to these. But yes, it's linear. Edit: Originally wrote f(x) = k·sin(k…
Depends on what space you define the derivative on. It is of course a bounded (and therefore continuous) operator from C^k to C^{k-1} for any positive integer K.
Additionally, it only really makes sense to talk about bounded operators between topological vectors spaces (as you need to make sense of what it means to be bounded), of which the most commonly dealt with are Banach spaces.
Re: I realized that derivatives are linear
#18This is why you take linear algebra and calculus before doing machine learning.
I used to teach fencing, in old days people would do months of footwork before being allowed to have a go at the actual sport. This changed because once the possibility of fatal encounters disappeared almost no one was motivated enough to go through the prerequisite.
But being clear, the footwork is the fundamental and can't be skipped.
Re: I realized that derivatives are linear
#19Well this is the whole point of derivatives (i.e. tangent maps): to be linear approximations of functions. So yes, a linear approximation of a linear function is the function itself.
You are right, but what you are saying has nothing to do with the author's point: what he is saying is that the differentiation operator itself is linear, which is a meaningful and true fact even in spaces where you have no idea of what a linear function is.
I think you'll find that derivatives are overwhelming defined as operators on vector spaces. It is definitely correct to talk about a linear function on a vector space.
Re: I realized that derivatives are linear
#20This is why you take linear algebra and calculus before doing machine learning.
On the contrary - ML is a great motivator to finally grapple the "prerequisites". During school I never understood what the math was for, so my unconscious brain never saw the necessity to actually learn it. Now I want to learn - with hugely better results. This mechanism should be utilized much more often instead of shoving seemingly unrelated knowledge into peoples ears without letting them feel the need for it fir…
Right. There are a lot of comments along the lines of "Duhhh, that's the point of calculus" here. But you know what? Screw that. People learn when people learn, and discovering it for yourself is a lot cooler than someone telling you about it, especially if it's related to investigating something you care about.
I can't tell you the number of 'trivial' math facts that I have (re)discovered because they were in the context of something I cared deeply about.
The point isn't to remember D_x is a linear operator--math isn't about memorization. It's about understanding the context where this is a useful fact and knowing how to figure it out.
Learn it in Calc I and you can half-heartedly reference it (...isn't differentiation linear? I feel like I remember that from senior year of high school...).
Figure it out on your own and you own it for life.
Post it to the internet and you get ridiculed and mocked for it so that you wish you could forget it.