Live data from Hacker News

I realized that derivatives are linear

codesmoothie.com

31–40 of 76 posts

Re: I realized that derivatives are linear

#31
This was an example used in my linear algebra class as soon as they started introducing the vector spaces in an abstract sense.

I think this post may still be too wedded to the idea of linear spaces and vectors being arrays of objects - specifically in insisting on decomposing functions like sin and cos to Taylor Series. In fact, you can have a vector space where, in addition to polynomial terms, there are also dimensions for sin(x), tan(x), sin(x - pi), e^x, etc. The fact that you can't enumerate these dimensions, or even describe the set of them until given a set of vectors you're trying to describe, doesn't keep this from being a vector space.

Re: I realized that derivatives are linear

#33
Be careful when using knowledge from this post. These are special cases, not the general rules of differentiaton (or calculus).

For example, for multi variable calculus, the results would be very different.

Let's take the example of W.X

d/dx (W.X) = X.d/dx(W) + W.d/dx(X)

since W is not dependent on x, the first term is zero and we get the answer the author got.

Before drawing conclusions from the post, please remember the assumptions the author has taken.

Re: I realized that derivatives are linear

#34
post #9

Well this is the whole point of derivatives (i.e. tangent maps): to be linear approximations of functions. So yes, a linear approximation of a linear function is the function itself.

If a function f passes through some point (a,b), then the tangent to f through that point is given by (y-b) = f'(a)·(x-a) and that function is affine but usually not linear. (For the tangent curve to be a linear function, you would need a·f'(a) = b, so that the tangent goes through the point (0,0).) It's not at all obvious to me that this means that the function d(f) = df/dx is linear. It is linear, but I don't see h…

The derivative is only the direction of the tangent line. The affine part does not come into play.

Re: I realized that derivatives are linear

#35

There seems to be a misconception that linear transformations have to look like lines.

One easy way to see that "linear" does not mean "line-like" is looking at matrix transforms: All matrix transforms are linear. And we can do a lot of stuff easily with those, like rotating, distorting, even perspective (with w-normalization).

Re: I realized that derivatives are linear

#36
post #31

This was an example used in my linear algebra class as soon as they started introducing the vector spaces in an abstract sense. I think this post may still be too wedded to the idea of linear spaces and vectors being arrays of objects - specifically in insisting on decomposing functions like sin and cos to Taylor Series. In fact, you can have a vector space where, in addition to polynomial terms, there are also dimen…

Hm. Interesting.

I always viewed real functions as infinite-dimensional vectors in the "canonical" basis, that is, shifted Dirac impulses. I guess it can be transformed into your representation with a change of basis with some handwaving.

Re: I realized that derivatives are linear

#37
So the point of this article is that differentiation is linear. That is, the operator D which takes f to d f/d x is linear. The author points out that one can write this down in as a matrix with respect to a basis of polynomials, which is nice for suitably well behaved functions and I think nice for understanding. Other operators one might look at are integration, Fourier or Laplace transforms, or more exotic integral transforms which are linear. One can view a Fourier transform like a change of basis.

In another sense, derivatives themselves are linear: for a function f: U -> V of vector spaces, the derivative (at some point) is a linear map from U -> V, (i.e. the derivative of the functions is a function Df: U -> L(U,V)) and this extends the concept of derivative to multiple dimensions as f(x+h) = f(x) + (Df)(x)h + o(h).

This seems ok at first derivatives but can become unwieldy as they became tensors higher rank.

Another question one might ask on learning that differntiation is a linear operator is what it’s eigenvalues are. For differentiation these are functions of the form f(x) = exp(ax). But one can construct other linear operators and from this you get Sturm–Liouville theory which is fantastic.

One final note is that much of this multidimensional derivatives and tensor stuff becomes a lot easier if one learns suffix notation (aka Einstein notation, aka index notation, aka summation convention), as well as perhaps a few identities with the kronecker delta or Levi-Civita symbol. Notation can break down a bit with arbitrary rank tensors: $a_{i_1,...,i_k}$ becomes unwieldy but writing $a_{pq...r}$ is ok.

Re: I realized that derivatives are linear

#38
post #36
post #31

This was an example used in my linear algebra class as soon as they started introducing the vector spaces in an abstract sense. I think this post may still be too wedded to the idea of linear spaces and vectors being arrays of objects - specifically in insisting on decomposing functions like sin and cos to Taylor Series. In fact, you can have a vector space where, in addition to polynomial terms, there are also dimen…

Hm. Interesting. I always viewed real functions as infinite-dimensional vectors in the "canonical" basis, that is, shifted Dirac impulses. I guess it can be transformed into your representation with a change of basis with some handwaving.

What you're describing may not be a subspace of the space I'm describing. Mine is definitely a subspace of yours - e.g. you can project functions into sums of shifted Dirac impulses by representing each function dimension as a linear combination of the "Dirac vectors" for that function's values.

Re: I realized that derivatives are linear

#39
HN continues to confuse me to no end.

Mention some mathematically advanced idea: out come the pitchforks about how you don't need that, all you need is code/market size/scalability/product fit/investment/execution.

Mention a banality that anyone who studied algebra knows: frontpage.

Re: I realized that derivatives are linear

#40
post #38
post #36

Earlier quoted context omitted.

Hm. Interesting. I always viewed real functions as infinite-dimensional vectors in the "canonical" basis, that is, shifted Dirac impulses. I guess it can be transformed into your representation with a change of basis with some handwaving.

What you're describing may not be a subspace of the space I'm describing. Mine is definitely a subspace of yours - e.g. you can project functions into sums of shifted Dirac impulses by representing each function dimension as a linear combination of the "Dirac vectors" for that function's values.

Hmmm. I think it’s easy to fix that:

Suppose that there’s a function f, that can be written as an infinite sum (integral) of shifted Dirac impulses, but cannot be written in your representation as a sum of those “base functions”. Then simply add a new dimension to your representation that will correspond to f, so that f will be represented as 1 at this new dimension, and zero everywhere else. (In other words: add f to the base functions)

Repeat until you have covered every function.

Post reply on HN