Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

I realized that derivatives are linear

codesmoothie.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: I realized that derivatives are linear

#41

Post by soVeryTired »

sampo wrote: post #5

The derivative is a linear operator, but it's not a bounded operator. That is, for example, the vector norm of f(x) = k·sin(x/k) → 0 when k→0, but the norm of d/dx f(x) does not. This also means that it's not continuous. Of the mappings between vector spaces, the most well behaving are the bounded linear operators, and the derivative doesn't belong to these. But yes, it's linear. Edit: Originally wrote f(x) = k·sin(k…

Anyone have a concrete example of an open subset of C^k whose preimage under differentiation is not open?

(Also, what's the most natural norm on C^k? The sup norm? I haven't done any functional analysis in years)

Re: I realized that derivatives are linear

#42

Post by wodenokoto »

The first headline for this was something along the lines of "I realized that derivatives are linear" making it clear that this is not a new discovery, but rather a person sharing a lightbulb moment.

I feel a lot of comments are saying "well of course they are!", not realizing that this is not about a new discovery.

Re: I realized that derivatives are linear

#43

Post by enriquto »

hgibbs wrote: post #17
sampo wrote: post #5

The derivative is a linear operator, but it's not a bounded operator. That is, for example, the vector norm of f(x) = k·sin(x/k) → 0 when k→0, but the norm of d/dx f(x) does not. This also means that it's not continuous. Of the mappings between vector spaces, the most well behaving are the bounded linear operators, and the derivative doesn't belong to these. But yes, it's linear. Edit: Originally wrote f(x) = k·sin(k…

Depends on what space you define the derivative on. It is of course a bounded (and therefore continuous) operator from C^k to C^{k-1} for any positive integer K. Additionally, it only really makes sense to talk about bounded operators between topological vectors spaces (as you need to make sense of what it means to be bounded), of which the most commonly dealt with are Banach spaces.

> Additionally, it only really makes sense to talk about bounded operators between topological vectors spaces

Reading your comment, I wondered how can you define bounded sets in a topological vector space (where you don't have a norm). The definition is cute: a set X if bounded if any neighborhood of 0 can be inflated to include the whole of X.

Re: I realized that derivatives are linear

#44

Post by leeeeech »

mduerksen wrote: post #8
anonytrary wrote: post #2

This is why you take linear algebra and calculus before doing machine learning.

On the contrary - ML is a great motivator to finally grapple the "prerequisites". During school I never understood what the math was for, so my unconscious brain never saw the necessity to actually learn it. Now I want to learn - with hugely better results. This mechanism should be utilized much more often instead of shoving seemingly unrelated knowledge into peoples ears without letting them feel the need for it fir…

calculus fundamentals pretty often lead to extremal problems. Those seem quite useful for optimization at any rate.

Re: I realized that derivatives are linear

#45

Post by 4ad »

chombier wrote: post #28
giomasce wrote: post #25

Earlier quoted context omitted.

I agree with you, but this is not what GGP is saying. GGP is saying another true fact (i.e., that derivative of a linear function is that same function), which a different thing than stating, as the article says, that the differentiation operator is linear. On a manifold there is no concept of a linear function, so you cannot say that the derivative of a linear function is the same function, but the differentiation o…

Of course you are correct, even though one could argue the linearity of differentiation is a property you can obtain by differentiating in coordinate charts , where the reasoning is still valid. In any case, it is probably a good thing to get a good intuition of what differentiation and derivatives are in the vector space setting before digging into differential geometry.

Not all vector spaces are manifolds (in fact most aren't), and you don't need charts to define differential operators (just see functional analysis).

Re: I realized that derivatives are linear

#46

Post by hansen »

soVeryTired wrote: post #41
sampo wrote: post #5

The derivative is a linear operator, but it's not a bounded operator. That is, for example, the vector norm of f(x) = k·sin(x/k) → 0 when k→0, but the norm of d/dx f(x) does not. This also means that it's not continuous. Of the mappings between vector spaces, the most well behaving are the bounded linear operators, and the derivative doesn't belong to these. But yes, it's linear. Edit: Originally wrote f(x) = k·sin(k…

Anyone have a concrete example of an open subset of C^k whose preimage under differentiation is not open? (Also, what's the most natural norm on C^k? The sup norm? I haven't done any functional analysis in years)

If you want Cᵏ to be a normed space you do

|f| = ∑ᵢ₌₀ᵏ sup|f⁽ⁱ⁾|.

On open domains you can also use the topology that forces uniform convergence on all compact subsets. But this will only give you a metric space, no Banach space (but you’ll include unbouded functions). This is needed for studying Brownian motions with an unbounded time domain.

Regarding the other question: If you take Cᵏ⁻¹ as a the co-domain differentiation will be continuous. IIRC to get unbounded linear maps defined on the whole Banach space you need the axiom of choice, you won’t be able to write one down.

The problem with differential operators is that they are usually only defined on a dense subset of the domain, and there they are not bounded. E.g. in quantum mechanics the space of states is L² but all the interesting observables are differential operators. You can weasel out of this situation by defining them on a subset of “physical states” (e.g. smooth wave functions of rapid decay). But they aren’t continuous anymore (the spectrum is unbounded). But on Hilbert spaces everything mostly works out fine. Physicist usually ignore those technical problems and still don’t make mistakes.

Re: I realized that derivatives are linear

#47

Post by adament »

hgibbs wrote: post #17
sampo wrote: post #5

The derivative is a linear operator, but it's not a bounded operator. That is, for example, the vector norm of f(x) = k·sin(x/k) → 0 when k→0, but the norm of d/dx f(x) does not. This also means that it's not continuous. Of the mappings between vector spaces, the most well behaving are the bounded linear operators, and the derivative doesn't belong to these. But yes, it's linear. Edit: Originally wrote f(x) = k·sin(k…

Depends on what space you define the derivative on. It is of course a bounded (and therefore continuous) operator from C^k to C^{k-1} for any positive integer K. Additionally, it only really makes sense to talk about bounded operators between topological vectors spaces (as you need to make sense of what it means to be bounded), of which the most commonly dealt with are Banach spaces.

It also depends on what topology you put on your space, while the derivative operator is defined on all of C^k, that does not make it continuous. In fact as the GPs example shows, the topology you put on C^k and C^{k-1} must be so that uniform convergence does not imply convergence in C^k (which differs from many peoples intuitive notion of convergence of functions) or cos(x/k) converges to 0 in C^{k-1} (which is just plain weird).

Re: I realized that derivatives are linear

#48

Post by enriquto »

tzahola wrote: post #11

Better later than never, I guess. I’ve heard that the next big thing will be this “chain rule”, you should check that out too.

hey, it is better to be nice in that case! Because you don't know anything when your are born, everyday thousands of people learn about well-known cool things that are new for them: https://www.xkcd.com/1053/

This guy even explained his rediscovery in a beautiful form.

Re: I realized that derivatives are linear

#49

Post by viraptor »

fwdpropaganda wrote: post #39

HN continues to confuse me to no end. Mention some mathematically advanced idea: out come the pitchforks about how you don't need that, all you need is code/market size/scalability/product fit/investment/execution. Mention a banality that anyone who studied algebra knows: frontpage.

You may be overestimating the banality. I studied CS with many math courses and some of the article goes over my head. (It would be clearer 10y ago) I don't expect many people here actually studied math as their main goal.

Re: I realized that derivatives are linear

#50

Post by fwdpropaganda »

viraptor wrote: post #49
fwdpropaganda wrote: post #39

HN continues to confuse me to no end. Mention some mathematically advanced idea: out come the pitchforks about how you don't need that, all you need is code/market size/scalability/product fit/investment/execution. Mention a banality that anyone who studied algebra knows: frontpage.

You may be overestimating the banality. I studied CS with many math courses and some of the article goes over my head. (It would be clearer 10y ago) I don't expect many people here actually studied math as their main goal.

That aside I think there's something bigger here.

I think learning is really hard work, and so most people's first reaction to hard work is to say No, and then go and construct an a posteriori rationale for why actually they shouldn't do that hard work (it's not that useful, you're never gonna use it, you're an expert at something else, etc).

Similar story for why asking data structures in job interviews is a bad idea when you're an applicant (but the people who have been hired and are hiring, do think it's good to ask)

Post reply on HN