A wonderful little video to understand what tensors are, by Daniel Fleish:
https://www.youtube.com/watch?v=f5liqUk0ZTw
Very simple and basic.
Edit: incorrectly wrote vectors instead of tensors.
Why tensors? A beginner's perspective
21–30 of 109 posts
Re: Why tensors? A beginner's perspective
#22I was happy to see that this article is actually talking about tensors, not just multidimensional arrays (which for some reasons are often called tensors by machine learning folks).
Same word, different contexts and meaning. In ML tensors are multidimensional arrays (and nothing more). Neither physicists nor ML researchers/developers are confused about what it means.
I am a bit confused
Re: Why tensors? A beginner's perspective
#23That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…
> Instead, author describes what mathematicians call “a tensor bundle”, which is a correspondence that assigns each point of space a unique tensor.
Technically, that’s a tensor field which is a section of the tensor bundle. Similarly, a vector field is a section of the tangent bundle (the collection of all the tangent spaces of the points on the manifold). A vector field is just a choice of a tangent vector for each point from that point’s tangent space.
Re: Why tensors? A beginner's perspective
#24Earlier quoted context omitted.
Since I also thought Tensors were just higher dimension arrays, isn't this really what ML folks think Tensors are, since they (we?) do attach units to the Tensors most of the time?
There are really just two things called tensors. Physicists' tensors = generalization of arrays with units; have to transform according to certain coordinate laws. Mathematicians' tensors = generalization of arrays, transformation rules don't matter.
But they are not totally different! The Physicists' tensors are actually Mathematicians' tensors, but parametrized by some parameters (coordinates). Then you have special laws what happens if you make a (possibly non-linear) change of the coordinates. See https://en.wikipedia.org/wiki/Tensor#Tensor_fields
Re: Why tensors? A beginner's perspective
#25Earlier quoted context omitted.
Since I also thought Tensors were just higher dimension arrays, isn't this really what ML folks think Tensors are, since they (we?) do attach units to the Tensors most of the time?
There are really just two things called tensors. Physicists' tensors = generalization of arrays with units; have to transform according to certain coordinate laws. Mathematicians' tensors = generalization of arrays, transformation rules don't matter.
> Mathematicians' tensors = generalization of arrays, transformation rules don't matter.
That's definitely inaccurate, at least it doesn't match what I think of as tensors in mathematics.
In mathematics, tensors are the most general result of a bilinear opteration. This does imply that they transform according to certain laws: if you represent the tensor using some particular basis, that basis can be expressed in the original vector spaces you multiplied, and choosing a different basis for your vector spaces results in a different basis for your tensors.
By "most general bilinear operation" I am talking about what is expressed in category theory as a universal property... with a morphism that preserves bilinear maps.
Tensors can be over multiple vector spaces or a single vector space (in which case it's typically implied that it's over the vector space and its dual). When you use a vector space and its dual, I believe you get the kind of tensor that physicists deal with, and all of the same properties. Note that while vector spaces and their dual may seem to be equivalent at first glance (and they are isomorphic in finite-dimensional cases), both mathematicians and physicists must know that they have different structure and transform differently.
Something that will throw you off is that mathematicians often like to use category theory and "point free" reasoning where you talk about vector spaces and tensor products in terms of things like objects and morphisms, and often avoid talking about actual vectors and tensors. Physicists talk about tensors using much more concrete terms and specify coordinate systems for them. It can require some insight in order to figure out that mathematicians and physicists are actually talking about the same thing, and figure out how to translate what a physicist says about a tensor to what a mathematician says.
Re: Why tensors? A beginner's perspective
#26My version is just: Tensors allow us to write data and operations on data in a way which does not depend on how we chose to represent them.
For example, if I have a vector x in V and a map T from V to W, then I would like the truth of T(x)=y to be independent of how I represent T and x.
Re: Why tensors? A beginner's perspective
#27That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…
To be a bit pedantic: the identification of tensors with multilinear forms requires finite dimensions (or reflexive topological spaces).
Re: Why tensors? A beginner's perspective
#28That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…
Why are you complaining that the author didn't talk about tensors as they are used in tensorflow? Tensorflow is never even mentioned in the piece. The author is perfectly clear in the first sentence that the piece's focus is about the usefulness of tensors in a physics context.
Huh, you’re right, not sure why I thought it’s TensorFlow related.
Re: Why tensors? A beginner's perspective
#29That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…
> author describes tensors as things behaving according to tensor transformation formula
In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick.
If you think developers are guilty of circular logic, check out some of the math pages on Wikipedia. You can get lost in moments.
Re: Why tensors? A beginner's perspective
#30I was happy to see that this article is actually talking about tensors, not just multidimensional arrays (which for some reasons are often called tensors by machine learning folks).
Same word, different contexts and meaning. In ML tensors are multidimensional arrays (and nothing more). Neither physicists nor ML researchers/developers are confused about what it means.
I'm pretty sure that a multidimensional array was the original implementation of tensors and the more fancy linear forms formalism came later in the twentieth century. Then the question was how do these multidimensional arrays operate on vectors and how do they transport around a manifold. (Source: first book I read on general relativity in the 1970s had a lot of pages about multidimensional arrays and also this good summary of the history: https://math.stackexchange.com/questions/2030558/what-is-the...
(Sort of like how vectors kind of got going via a list of numbers and then they found the right axioms for vector spaces and then linear algebra shifted from a lot of computation to a sort of spare and elegant set of theorems on linearity).