Lobachevsky... "the analytic and algebraic topology of locally Euclidean metrizations of infinitely differentiable Riemannian manifolds"
боже мой.
> боже мой.
Pokemon?
What is a manifold?
101–110 of 138 posts
Re: What is a manifold?
#102This is a very informative article about the history of manifolds and their significance. Don’t let the title fool you into this being just a definition. It’s actually much more well written than the majority or articles we usually come across.
Is that really a good article? I thought it was average. It had some big flaws but was probably still informative for readers with no mathematical knowledge in the domain. For instance, consider the only concrete example in the article: the space of all possible configurations of a double pendulum is a manifold. The author claims it's useful to see it in a manifold, but why? Precisely, why more as a manifold than as…
Minkowski spacetime is the term in special relativity, i.e. the flat case, or zero curvature. In general relativity, spacetime is a pseudo Riemannian manifold, like the sibling comment says. Unlike Minkowski spacetime, it can be curved.
Re: What is a manifold?
#103What a terrible article. Can anyone who is not a mathematician tell me one thing they learned from this? The naked term "manifold" in its modern usage, refers to a topological manifold, loosely a locally euclidean hausdorff topological space, which has no geometry intrinsic to it at all. The hyperbolic plane and the euclidean plane are different geometries you can put on the same topological manifold, and even does n…
> Can anyone who is not a mathematician tell me one thing they learned from this?
I can share my two take-aways.
- in the geometric sense, manifolds are spaces analogous to curved 2d surfaces in 3d that extend to an arbitrary number of dimensions
- manifolds are locally Euclidean
If I were to extrapolate from the above, i'd say that:
- we can map a Euclidean space to every point on a manifold and figure out the general transformation rules that can take us from one point's Euclidean space to another point's.
- manifolds enable us to discuss curved spaces without looking at their higher-dimension parent spaces (e.g. in the case of a sphere surface we can be content with just two dimensions without working in 3d).
Naturally, I may be totally wrong about all this since I have no knowledge on the subject...
Re: What is a manifold?
#104Does the way "manifold" is used when describing subsets of the representational space of neural networks (e.g. "data lies on a low-dimensional manifold within the high-dimensional representation space") actually correspond to this formal definition, or is it just co-opting the name to mean something simpler (just an embedded sub-space)? If it is the formal definition being used, then why? Do people actually reason ab…
It's hard to prove rigorously which is why people usually refer to it as the "manifold hypothesis." But it is reasonable to suppose that (most) data does live on a manifold in the strict sense of the term. If you imagine the pixels associated with a handwritten "6", you can smoothly deform the 6 into a variety of appearances where all the intermediate stages are recognizable as a 6.
However the embedding space of a typical neural network that is representing the data is not a manifold. If you use ReLU activations the kinks that the ReLU function creates break the smoothness. (Though if you exclusively used a smooth activation function like the swish function you could maintain a manifold structure.)
Re: What is a manifold?
#105This reminds me of how physicists will define a tensor. So a second rank tensor is the object that transforms according as second rank tensor when the basis (or coordinates) changes. You might find it circular reasoning but it is not, This transformation property is what distinguishes tensors (of any rank) from mere arrays of numbers. Looking at things from abstract view does allow us not to worry about how we visual…
This is a tendency among physicists that I find a bit painful when reading their explanations: focusing on how things transform between coordinate systems rather than on the coordinate-independent things that are described by those coordinates. I get that these transformation properties are important for doing actual calculations, but I think they tend to obfuscate explanations. In special relativity, for example, a…
Taylor & Wheeler's Spacetime Physics is similar. They emphasize the importance of frame invariant representations. (I highly recommend the first edition over the second edition, the second edition was a massive downgrade.)
Kip Thorne was also heavily influenced by this geometric approach. Modern Classical Physics by Thorne & Blandford uses a frame invariant, geometric approach throughout, which (imo) makes for much simpler and more intuitive representations. It allows you to separate out the internal physics from the effect of choosing a particular coordinate system.
Re: What is a manifold?
#106Does the way "manifold" is used when describing subsets of the representational space of neural networks (e.g. "data lies on a low-dimensional manifold within the high-dimensional representation space") actually correspond to this formal definition, or is it just co-opting the name to mean something simpler (just an embedded sub-space)? If it is the formal definition being used, then why? Do people actually reason ab…
The closest thing that you may get is a manifold + noise. Maybe some people thing about it in that way. Think for example of the graph of y=sin(x)+noise, you can say that this is a 1 dimensional data manifold. And you can say that locally a data manifold is something that looks like a graph or embedding (with more dimensions) plus noise.
But i am skeptical whether this definition can be useful in the real world of algorithms. For example you can define things like topological data analysis, but the applications are limited, mainly due to the curse of dimensionality.
Re: What is a manifold?
#107This is a very informative article about the history of manifolds and their significance. Don’t let the title fool you into this being just a definition. It’s actually much more well written than the majority or articles we usually come across.
Is that really a good article? I thought it was average. It had some big flaws but was probably still informative for readers with no mathematical knowledge in the domain. For instance, consider the only concrete example in the article: the space of all possible configurations of a double pendulum is a manifold. The author claims it's useful to see it in a manifold, but why? Precisely, why more as a manifold than as…
> Precisely, why more as a manifold than as a square
In a double pendulum, each arm can freely rotate (there is no stopping point). This means 0 degrees and 360 degrees are the same point, so the edges of the square are actually joined. If you join the left and right edges to each other, then join the top and bottom edges to each other, you end up with a torus.
Re: What is a manifold?
#108This reminds me of how physicists will define a tensor. So a second rank tensor is the object that transforms according as second rank tensor when the basis (or coordinates) changes. You might find it circular reasoning but it is not, This transformation property is what distinguishes tensors (of any rank) from mere arrays of numbers. Looking at things from abstract view does allow us not to worry about how we visual…
I don't get why people act like this definition is so circular. If you were to explain in detail what "transforms as a second rank tensor" means then it wouldn't be circular anymore. This just isn't the full definition.
Re: What is a manifold?
#109Does the way "manifold" is used when describing subsets of the representational space of neural networks (e.g. "data lies on a low-dimensional manifold within the high-dimensional representation space") actually correspond to this formal definition, or is it just co-opting the name to mean something simpler (just an embedded sub-space)? If it is the formal definition being used, then why? Do people actually reason ab…
There's a field known as information geometry. I don't know much about it myself as I'm more into physics, but here's a recent example of applying geometrical analysis to neural networks. Looks interesting as they find a phenomenon analogous to phase transitions during training
Information Geometry of Evolution of Neural Network Parameters While Training
Re: What is a manifold?
#110Earlier quoted context omitted.
To be fair to physicists, the standard physicists' definition isn't "a tensor is a thing that transforms like a tensor", it's "a tensor is a mathematical object that transforms in the following way ". When people say "a tensor is a thing that transforms like a tensor" they're using a convenient shorthand for the bit that I put in angle brackets above. My favourite explanation is that "Tensors are the facts of the uni…
Right, but if you fill in the shorthand there’s no reason to think it’s circular; it’s just a normal definition at that point, albeit one without much motivation.
But it's not possible to fill in the shorthand unless you already know what it stands for. Hence: the shorthand is not useful for communicating information, only for social signaling.