Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why tensors? A beginner's perspective

mfaizan.github.io

Post reply on HN
Print view View original

71–80 of 109 posts

Re: Why tensors? A beginner's perspective

#71

Post by ravi-delia »

jhokanson wrote: post #31
hinkley wrote: post #29

Earlier quoted context omitted.

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pa…

Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

What are we calling advanced math? There comes a point where I personally find it much easier to avoid examples until I'm problem-solving, since otherwise I'll get stuck in a loop of wondering if the thing I noticed generalizes. Could just be that my working memory is poor, but when I see a real honest number I know I'm in for a grueling day.

Re: Why tensors? A beginner's perspective

#72

Post by ravi-delia »

kaashif wrote: post #59
ericphanson wrote: post #3

I was happy to see that this article is actually talking about tensors, not just multidimensional arrays (which for some reasons are often called tensors by machine learning folks).

It's not clear to me what you're annoyed about exactly. The way I see it, there are a few options: You're getting annoyed that people are confusing the map with the territory [1]. Multidimensional arrays with certain properties can be used to represent tensors, but aren't tensors. In the same way a diagram of torus isn't a topological space, or a multiplication table isn't a group, or a matrix is not a linear map. Is…

I mean, if I wanted to refer to the reals exclusively as a vectorspace I wouldn't be wrong, but if you aren't actually using what makes it a vectorspace why would you choose to call it that? Hell, call 7 a tensor. There's more than a map-territory distinction (I'd argue formal mathematics is perhaps the only realm where the two are one and the same, but I see what you're saying), it's a convention of language more generally. You typically use the most necessary term, rather than a random also accurate label. If you don't care about invariance under coordinate transformations (and most machine learning does not), why would you call it a tensor?

Re: Why tensors? A beginner's perspective

#73

Post by ravi-delia »

uoaei wrote: post #40
icapybara wrote: post #6

Earlier quoted context omitted.

Tensors have additional properties that arrays don't necessarily have. For example, the coordinate system transform rule that the author describes in the beginning of the post. One of my old physics professors taught us to think of tensors as "arrays with units." If it's a vector/matrix/higher dimensional array but has physical units, it's probably a tensor. The fact that it has units means it represents something ph…

This is a pretty tired line, I gotta say. Obviously they are not. One is a linear operator, the other is a data structure for implementing computations using that operator. This description extends to all tensors. It's like saying "queues are not just lists". That is true and also neither insightful nor helpful. I don't see it as mystifying or complicated, what am I missing?

The reason we carve out a category of tensor, even when you could in principle just define ad-hoc functions on vectors and call it a day, is that we notice the commonalities between a number of objects which are invariant with respect to coordinate changes. Machine learning generally does not use this invariance at all, and has arrays which happen to be tensors for largely unrelated reasons. Calling them tensor makes more sense than calling, say, real numbers tensors, but less sense than calling reals reals.

Re: Why tensors? A beginner's perspective

#74

Post by tagrun »

contravariant wrote: post #46
tagrun wrote: post #42

Earlier quoted context omitted.

It has nothing to do with tensor fields, uniform/constant tensors still obey the proper coordinate transformations, that's the defining property of any tensor. (With non-uniform tensor fields, covariant derivatives also pick up a correction, but that's a separate thing.) TensorFlow "tensor"(and most other use of "tensor" in programmer jargon) is not a tensor at all, it's just a multidimensional array.

Mathematicians would disagree with you there. There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Matrices (or linear transformations in general) are important examples of tensors. There's a nice adjunction between tensor spaces A(x)B and the space of linear transformations B=>C given by: Hom(A(x)B, C) = Hom(A, B=>C) In the case…

Which mathematician disagrees with what exactly?

Tensors are introduced by physicists to ensure various physical quantities (which involve coordinates and their derivatives) do not depend on the arbitrarily chosen coordinate system. This is ensured through the transformation properties of tensors.

The name tensor itself comes from the theory of elasticity, Cauchy stress tensor, which BTW is uniform in many practical cases, and obeys the following tensor transformation rule:

https://en.wikipedia.org/wiki/Cauchy_stress_tensor#Transform...

like any other (contravariant) tensor must.

Matrices are not examples of tensors. Matrices can be used for representation of tensors, in which case tensor product becomes Kronecker product, but matrices in general don't have to represent tensors. You can put anything, including your favorite colors or a list of random numbers, in a matrix, and it won't be a tensor in general, not unless it must transform like a tensor under coordinate system changes.

Similarly, TensorFlow "tensor" is just a multidimensional data array, with no transformation rules enforced on it, and therefore is not a tensor.

Re: Why tensors? A beginner's perspective

#75

Post by mhh__ »

lalaithion wrote: post #33
ericphanson wrote: post #3

I was happy to see that this article is actually talking about tensors, not just multidimensional arrays (which for some reasons are often called tensors by machine learning folks).

This is mostly a semantic argument, but I find this to be a very annoying perspective. Given a basis, there is a natural isomorphism between tensors of a certain type and multidimensional arrays of certain dimensions.

But it's like calling my table a cat because they both have four legs.

Re: Why tensors? A beginner's perspective

#76

Post by bigger_cheese »

xyzzyz wrote: post #14

That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…

Engineering usage seems to match the physics usage. In classic engineering fashion however we were always taught just to 'plug them in' without learning all the minutia that go with them.

For example the stress and strain calculations which are used for calculating Deformation (Say if you were rolling a sheet of steel in a mill) makes use of tensors and also something called an "Invariant" I assume this also comes from Physics/Mathematics world.

Re: Why tensors? A beginner's perspective

#78

Post by vcxy »

tagrun wrote: post #74
contravariant wrote: post #46

Earlier quoted context omitted.

Mathematicians would disagree with you there. There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Matrices (or linear transformations in general) are important examples of tensors. There's a nice adjunction between tensor spaces A(x)B and the space of linear transformations B=>C given by: Hom(A(x)B, C) = Hom(A, B=>C) In the case…

Which mathematician disagrees with what exactly? Tensors are introduced by physicists to ensure various physical quantities (which involve coordinates and their derivatives) do not depend on the arbitrarily chosen coordinate system. This is ensured through the transformation properties of tensors. The name tensor itself comes from the theory of elasticity, Cauchy stress tensor, which BTW is uniform in many practical…

Interesting, you use the word matrix differently than me. The way I use it, a 2d array isn't necessarily a matrix. It's only a matrix if it represents a linear map between vector spaces with respect to chosen bases. Then again, I'm not much of a programmer, but I've taught linear algebra a few times. My head is just in a different place I guess.

Re: Why tensors? A beginner's perspective

#79

Post by [deleted] »

tagrun wrote: post #74
contravariant wrote: post #46

Earlier quoted context omitted.

Mathematicians would disagree with you there. There are no coordinates to transform in an ordinary tensor space and therefore no way for a tensor to be affected by such a transformation. Matrices (or linear transformations in general) are important examples of tensors. There's a nice adjunction between tensor spaces A(x)B and the space of linear transformations B=>C given by: Hom(A(x)B, C) = Hom(A, B=>C) In the case…

Which mathematician disagrees with what exactly? Tensors are introduced by physicists to ensure various physical quantities (which involve coordinates and their derivatives) do not depend on the arbitrarily chosen coordinate system. This is ensured through the transformation properties of tensors. The name tensor itself comes from the theory of elasticity, Cauchy stress tensor, which BTW is uniform in many practical…

[deleted]

Re: Why tensors? A beginner's perspective

#80

Post by cyber_kinetist »

brummm wrote: post #16
saberience wrote: post #2

This doesn't seem like it's for beginners.

Hmm, this is stuff physicists learn in their first year undergrad classes for mathematical foundations. Seems to me it's the very definition of beginner.

I don't know what undergraduate program you have gone through, but this is definitely second-year or third-year course material for most physics degrees in universities. Maybe if you've already taken lots of AP classes in high school then you might be able to skip some stuff, but we're talking about the standard curriculum here.

Normally, you first study the distinction between vectors (which can be expanded to tensors) and scalars in second-year Analytical Mechanics class. You also get a taste of tensors toward the later material in Electromagnetism (which is also probably second-year). And you finally arrive at a rigorous definition of tensors when you take Mathematical Physics (second-year or third-year depending on your skills).

Post reply on HN