Speaking of: https://www2.math.upenn.edu/~ghrist/notes.html
Deep Learning Is Applied Topology
21–30 of 200 posts
Re: Deep Learning Is Applied Topology
#22If it was topology we wouldn't bother to warp the manifold so we can do similarity search. No, it's geometry , with a metric. Just as in real life, we want to be able to compare things. Topological transformation of the manifold happens during training too. That makes me wonder: how does the topology evolve during training? I imagine it violently changing at first before stabilizing, followed by geometric refinement.…
Agree, if anything it's Applied Linear Algebra...but that sounds less exotic.
Re: Deep Learning Is Applied Topology
#23[deleted]
Re: Deep Learning Is Applied Topology
#24Just because manifold looks a bit like burrito if you squint doesn't mean it is a burrito.
What if you don't have to squint very much?
Re: Deep Learning Is Applied Topology
#25Earlier quoted context omitted.
I cannot understand this prideful resentment of theory common among self-described practitioners. Even if existing theory is inadequate, would an operating theory not be beneficial? Or is the mystique combined with guess&check drudgery job security?
Maybe a little less with the ad hominems? The OP is providing an accurate description of an extremely immature field.
Many mathematicians are (rightly, IMO) allergic to assertions that certain branches are not useful (explicit in OP) and especially so if they are dismissive of attempts to understand complicated real world phenomema (implicit in OP, if you ask me).
Re: Deep Learning Is Applied Topology
#26Thanks for sharing. I also tend to view learning in terms of manifolds. It's a powerful representation.
> I'm personally pretty convinced that, in a high enough dimensional space, this is indistinguishable from reasoning
I actually have journaled extensively about this and even written some on Hacker News about it with respect to what I've been calling probabilistic reasoning manifolds:
> This manifold is constructed via learning a decontextualized pattern space on a given set of inputs. Given the inherent probabilistic nature of sampling, true reasoning is expressed in terms of probabilities, not axioms. It may be possible to discover axioms by locating fixed points or attractors on the manifold, but ultimately you're looking at a probabilistic manifold constructed from your input set.
> But I don't think you can untie this "reasoning" from your input data. It's possible you will find "meta-reasoning", or similar structures found in any sufficiently advanced reasoning manifold, but these highly decontextualized structures might be entirely useless without proper recontextualization, necessitating that a reasoning manifold is trained on input whose patterns follow learnable underlying rules, if the manifold is to be useful for processing input of that kind.
> Decontextualization is learning, decomposing aspects of an input into context-agnostic relationships. But recontextualization is the other half of that, knowing how to take highly abstract, sometimes inexpressible, context-agnostic relationships and transform them into useful analysis in novel domains
Full comment: https://news.ycombinator.com/item?id=42871894
Re: Deep Learning Is Applied Topology
#27Isn't it more differential geometry?
Re: Deep Learning Is Applied Topology
#28Data doesn't actually live on a manifold. It's an approximation used for thinking about data. Near total majority, if not 100%, of the useful things done in deep learning have come from not thinking about topology in any way. Deep learning is not applied anything, it's an empirical field advanced mostly by trial and error and, sure, a few intuitions coming from theory (that was not topology).
I disagree with this wholeheartedly. Sure, there is lots of trial and error, but it’s more an amalgamation of theory from many areas of mathematics including but not limited to: topology, geometry, game theory, calculus, and statistics. The very foundations (i.e. back-propagation) is just the chain rule applied to the weights. The difference is that deep learning has become such an accessible (sic profitable) field t…
Can you give an example where theories and techniques from other fields are reinvented? I would be genuinely interested for concrete examples. Such "reinventions" happen quite often in science, so to some degree this would be expected.
Re: Deep Learning Is Applied Topology
#29The title, as it stands, is trite and wrong. More about that a little later. The article on the other hand is a pleasant read.
Topology is whatever little structure that remains in geometry after you throwaway distances, angles, orientations and all sorts of non tearing stretchings. It's that bare minimum that still remains valid after such violent deformations.
While notion of topology is definitely useful in machine learning, -- scale, distance, angles etc., all usually provide lots of essential information about the data.
If you want to distinguish between a tabby cat and a tiger it would be an act of stupidity to ignore scale.
Topology is useful especially when you cannot trust lengths, distances angles and arbitrary deformations. That happens, but to claim deep learning is applied topology is absurd, almost stupid.
Re: Deep Learning Is Applied Topology
#30Data doesn't actually live on a manifold. It's an approximation used for thinking about data. Near total majority, if not 100%, of the useful things done in deep learning have come from not thinking about topology in any way. Deep learning is not applied anything, it's an empirical field advanced mostly by trial and error and, sure, a few intuitions coming from theory (that was not topology).
It’s alchemy. Deep learning in its current form relates to a hypothetical underlying theory as alchemy does to chemistry. In a few hundred years the Inuktitut speaking high schoolers of the civilisation that comes after us will learn that this strange word “deep learning” is a left over from the lingua franca of yore.
Not really, most of the current approaches are some approximations of the partition function.