Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is a manifold?

quantamagazine.org

Post reply on HN
Print view View original

111–120 of 138 posts

Re: What is a manifold?

#111

Post by lisper »

seanhunter wrote: post #84
lisper wrote: post #76

Earlier quoted context omitted.

> a convenient shorthand for the bit that I put in angle brackets above. Yes, but the "convenient shorthand" only makes sense if you already know what a tensor is. That renders the "definition" useless as an explanation or as pedagogy. It's only useful as a social signal to let others know that you understand what a tensor is (or at least you think you do). > My favourite explanation is that "Tensors are the facts of…

> But I would go with something more like: tensors are a way to represent vectors so that the representation of a given vector is the same no matter what basis (or coordinate system) you choose for your vector space. That's just incorrect though for a couple of reasons. Firstly, a vector in the sense in which it is used in physics is a rank 1 tensor so it has this transformation behaviour just like other higher order…

> That's just incorrect though

Quite possible. But that's in no small measure because I have yet to find an actual cogent definition of "tensor" that distinguishes a tensor from an array. (I have a similar problem with monads.)

> So what I mean when I talk about the reality of the tensor I mean whatever it is the tensor is expressing in the physical universe

OK, but then "the reality of a tensor" not depending on the coordinate system has nothing to do with tensors, and becomes a vacuous observation. It is simply a fact that actual physical quantities don't depend on how you write them down, and hence don't change when you write them down in different ways.

Re: What is a manifold?

#112

Post by qbit42 »

youoy wrote: post #106
HarHarVeryFunny wrote: post #99

Does the way "manifold" is used when describing subsets of the representational space of neural networks (e.g. "data lies on a low-dimensional manifold within the high-dimensional representation space") actually correspond to this formal definition, or is it just co-opting the name to mean something simpler (just an embedded sub-space)? If it is the formal definition being used, then why? Do people actually reason ab…

The closest thing that you may get is a manifold + noise. Maybe some people thing about it in that way. Think for example of the graph of y=sin(x)+noise, you can say that this is a 1 dimensional data manifold. And you can say that locally a data manifold is something that looks like a graph or embedding (with more dimensions) plus noise. But i am skeptical whether this definition can be useful in the real world of al…

Sometimes statistical rates for empirical risk minimization can be related to the intrinsic dimension of the data manifold (and noise level if present). In such cases, you are running the same algorithm but getting a performance guarantee that depends on the structure of the data, stronger when it is low dimensional.

Re: What is a manifold?

#113

Post by Iwan-Zotow »

getnormality wrote: post #21
robot-wrangler wrote: post #12

Earlier quoted context omitted.

I'm always surprised more people don't know about Quanta. Seems like it's currently the best science journalism out there, and IMO a very strong candidate for the single best place on the internet that's not crowd-sourced. The mixture of original art and technical diagrams is outstanding. Podcast is pretty good too, but I do wish they'd expand it to have someone with a good voice reading all the articles. Besides not…

It's because of their Simons Foundation support, but not only because of that. I mean, I invite anyone to name another billionaire pet project of comparable quality.

Clay Mathematics Institute, with its 7 Millennium problems

Re: What is a manifold?

#114

Post by Iwan-Zotow »

robotresearcher wrote: post #70
getnormality wrote: post #21

Earlier quoted context omitted.

It's because of their Simons Foundation support, but not only because of that. I mean, I invite anyone to name another billionaire pet project of comparable quality.

Carnegie Libraries, Nobel Prizes, Rhodes scholarships?

Clay Mathematics Institute, with its 7 Millennium problems

Re: What is a manifold?

#115

Post by mmooss »

senderista wrote: post #98
mmooss wrote: post #88

Earlier quoted context omitted.

This part I don't grasp: > this global coordinate system isn't a continuous mapping (see the discontinuity of both angular coordinates between 2*pi and 0). I'm guessing that the issue is that I don't know your definition of 'continuous'. I believe every point on the planet (sphere, for simplification) has unique corresponding coordinates on the map projection (chart). The only exceptions I can see are, A) surfaces pe…

Continuity is fundamentally a topological property of a mapping. It just means that for a mapping F and a point p, for any neighborhood del of F(p), we can find a neighborhood eps of p such that F(eps) is contained entirely in del. In simpler terms, if you draw a little ball around F(p), I can find a little ball around p whose image under F is contained in the little ball you drew around F(p). If I have coordinates o…

That makes sense. As I thought, I just needed to understand continuity in this context. That also helps address my original question - why manifolds aren't widely used in cartography. Thank you.

Re: What is a manifold?

#117

Post by senderista »

omnicognate wrote: post #85
codethief wrote: post #81

Earlier quoted context omitted.

Weinberg ≠ Wald. Wald's book is great! (For GR, of course, not SR.)

Indeed! I meant that it's good to know Wald is mathematically modern and not encrusted with coordinates. Saves me buying another book :-D (The comment I replied to mentioned both.)

I think it does a very good job of explaining the abstract index notation, which is superficially similar to coordinate notation but conceptually quite different.

Re: What is a manifold?

#118

Post by macleginn »

antognini wrote: post #104
HarHarVeryFunny wrote: post #99

Does the way "manifold" is used when describing subsets of the representational space of neural networks (e.g. "data lies on a low-dimensional manifold within the high-dimensional representation space") actually correspond to this formal definition, or is it just co-opting the name to mean something simpler (just an embedded sub-space)? If it is the formal definition being used, then why? Do people actually reason ab…

It's hard to prove rigorously which is why people usually refer to it as the "manifold hypothesis." But it is reasonable to suppose that (most) data does live on a manifold in the strict sense of the term. If you imagine the pixels associated with a handwritten "6", you can smoothly deform the 6 into a variety of appearances where all the intermediate stages are recognizable as a 6. However the embedding space of a t…

People also apply the notion of data manifold to language data (which is fundamentally discrete), and even for images the smoothness is hard to come buy (e.g., "images of cars" is not smooth because of shape and colour discontinuities). I guess the best we can do is to hope that there is an underlying virtual "data manifold" from which our datapoints have been "sampled", and knowing its structure may be useful.

Re: What is a manifold?

#119

Post by aleph_minus_one »

mulmen wrote: post #74
aleph_minus_one wrote: post #55

Earlier quoted context omitted.

Often, if the concept is presented in a more complex way the reason is that the author wants to emphasize and explain how the concept relates in a non-trivial way to some other deep concept; thus you learn a lot more than when the author explains things in the most simple (and shallow) way.

I don’t see how that is beneficial. If a simple concept relates to a complex one then explain the complexity, don’t add it.

> If a simple concept relates to a complex one then explain the complexity, don’t add it.

So write a text of at least 500 pages to explain the complexity. :-)

Re: What is a manifold?

#120

Post by hansvm »

macleginn wrote: post #118
antognini wrote: post #104

Earlier quoted context omitted.

It's hard to prove rigorously which is why people usually refer to it as the "manifold hypothesis." But it is reasonable to suppose that (most) data does live on a manifold in the strict sense of the term. If you imagine the pixels associated with a handwritten "6", you can smoothly deform the 6 into a variety of appearances where all the intermediate stages are recognizable as a 6. However the embedding space of a t…

People also apply the notion of data manifold to language data (which is fundamentally discrete), and even for images the smoothness is hard to come buy (e.g., "images of cars" is not smooth because of shape and colour discontinuities). I guess the best we can do is to hope that there is an underlying virtual "data manifold" from which our datapoints have been "sampled", and knowing its structure may be useful.

Those are less problematic than you might imagine.

- For language, individual words might be discrete, but concepts being communicated have more nuance and fill in the gaps.

- For language, even to the extent that discreteness applies, you can treat the data as being sampled from a coarser manifold and still extract a lot of meaningful structure.

- Images of cars are more continuous than you might imagine because of hue differences induced by time of day, camera lens, shadows, etc.

- Images of cars are potentially smooth even when considering shape and color discontinuities. Manifolds don't have to be globally connected. Local differentiability is usually the thing people are looking for in practical applications.

Post reply on HN