Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why tensors? A beginner's perspective

mfaizan.github.io

Post reply on HN
Print view View original

31–40 of 109 posts

Re: Why tensors? A beginner's perspective

#31

Post by jhokanson »

hinkley wrote: post #29
xyzzyz wrote: post #14

That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pa…

Speaking of math pages on Wikipedia ... and math text more generally

Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

Re: Why tensors? A beginner's perspective

#32

Post by chobytes »

hinkley wrote: post #29
xyzzyz wrote: post #14

That was explanation from a perspective of someone acquainted with modern physics. As such, it will make sense to physicist, but no sense to most everyone else, including mathematicians who don’t know modern physics. For example, in the beginning, author describes tensors as things behaving according to tensor transformation formula. This is already very much a physicist kind of thinking: it assumes that there is som…

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pa…

I think a lot of circularity occurs in mathematics because we don't typically qualify our utterances when it can be implicitly understood.

Eg "Numbers (formal) are those objects which behave like numbers (informal)."

Re: Why tensors? A beginner's perspective

#33

Post by lalaithion »

ericphanson wrote: post #3

I was happy to see that this article is actually talking about tensors, not just multidimensional arrays (which for some reasons are often called tensors by machine learning folks).

This is mostly a semantic argument, but I find this to be a very annoying perspective. Given a basis, there is a natural isomorphism between tensors of a certain type and multidimensional arrays of certain dimensions.

Re: Why tensors? A beginner's perspective

#34

Post by westurner »

lanstin wrote: post #30
jstx1 wrote: post #12

Earlier quoted context omitted.

Same word, different contexts and meaning. In ML tensors are multidimensional arrays (and nothing more). Neither physicists nor ML researchers/developers are confused about what it means.

I'm pretty sure that a multidimensional array was the original implementation of tensors and the more fancy linear forms formalism came later in the twentieth century. Then the question was how do these multidimensional arrays operate on vectors and how do they transport around a manifold. (Source: first book I read on general relativity in the 1970s had a lot of pages about multidimensional arrays and also this good…

AFAIU, Matrices are categorically subsets of Tensors where the product operator, at least, is not the tensor product but the Dot product.

Dot product: https://en.wikipedia.org/wiki/Dot_product

Matrix multiplication > Dot product, bilinear form and inner product: https://en.wikipedia.org/wiki/Matrix_multiplication#Dot_prod...

> The dot product of two column vectors is the matrix product

Tensor > Geometric objects https://en.wikipedia.org/wiki/Tensor :

> The transformation law for a tensor behaves as a functor on the category of admissible coordinate systems, under general linear transformations (or, other transformations within some class, such as local diffeomorphisms.) This makes a tensor a special case of a geometrical object, in the technical sense that it is a function of the coordinate system transforming functorially under coordinate changes.[24] Examples of objects obeying more general kinds of transformation laws are jets and, more generally still, natural bundles.[25][26]

Re: Why tensors? A beginner's perspective

#35

Post by chobytes »

jhokanson wrote: post #31
hinkley wrote: post #29

Earlier quoted context omitted.

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pa…

Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

This is definitely a problem! Having a large set of interests and problems to draw examples and intuition from are how I deal with it. I suspect this is why so many mathematicians are also into physics.

Re: Why tensors? A beginner's perspective

#36

Post by exdsq »

chobytes wrote: post #35
jhokanson wrote: post #31

Earlier quoted context omitted.

Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

This is definitely a problem! Having a large set of interests and problems to draw examples and intuition from are how I deal with it. I suspect this is why so many mathematicians are also into physics.

100%! For those of us who need to learn from practical examples through to generalized intuition maths can be really really hard to learn depending on the source. Wish I was one of those people who finds it easier to learn from abstract first through to implementations second.

Re: Why tensors? A beginner's perspective

#37

Post by hinkley »

jhokanson wrote: post #31
hinkley wrote: post #29

Earlier quoted context omitted.

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pa…

Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

Randall Monroe has a comic about how most people need enough math to be able to handle a birthday dinner where the guests split the bill for the birthday boy/girl evenly and pay for their meals and tip separately.

That’s a pretty good bar and I wonder if we could just cut to that chase earlier. But I also believe that people need enough math to see when they’re being cheated, and I feel like you could just tell middle schoolers that and they would pay attention. Maybe even primary school.

You told Billy he could have three apples, and now there are two left. Did Billy take more apples than he should have?

It’s always how do you share your cookies fairly with your friends and if they’re my cookies why do I have to share them at all? Screw “fairly” I’m keeping the extras at least. That sort of sharing is a socially advanced concept they don’t entirely get just yet.

Re: Why tensors? A beginner's perspective

#38

Post by GoatOfAplomb »

jhokanson wrote: post #31
hinkley wrote: post #29

Earlier quoted context omitted.

> author describes tensors as things behaving according to tensor transformation formula In grade school it drove me nuts when the homework required us to describe a word without using the word (or it’s Latinate siblings). And yet as an adult there are few enough weeks that go by where some grownup doesn’t try to pull that same trick. If you think developers are guilty of circular logic, check out some of the math pa…

Speaking of math pages on Wikipedia ... and math text more generally Is it just me or are we horrible at teaching advanced math? Where are the examples (with actual numbers)? Where is the motivation? Where are the pictures?

> Where are the examples (with actual numbers)?

In upper-level undergraduate math, I made a game of seeing how many pages I would go before seeing 7 printed anywhere. It was usually 10 pages, if I included the page numbers.

Re: Why tensors? A beginner's perspective

#39

Post by russellbeattie »

cloogshicer wrote: post #17
jstx1 wrote: post #12

Earlier quoted context omitted.

Same word, different contexts and meaning. In ML tensors are multidimensional arrays (and nothing more). Neither physicists nor ML researchers/developers are confused about what it means.

> Neither physicists nor ML researchers/developers are confused about what it means. I'm sure this has confused a lot of people, especially beginners. Clashing terminology is one of the main difficulties in interdisciplinary work, in my experience. I don't think it's good to shrug it off like that.

I'm literally learning this, or at least being reminded of something I had totally forgotten, as I read this thread. So yeah.

Words can have multiple meanings, but I think we can all agree it's preferable if they don't have multiple slightly different depending on context meanings. That's just confusing.

Re: Why tensors? A beginner's perspective

#40

Post by uoaei »

icapybara wrote: post #6
The_rationalist wrote: post #5

Earlier quoted context omitted.

What is the difference?

Tensors have additional properties that arrays don't necessarily have. For example, the coordinate system transform rule that the author describes in the beginning of the post. One of my old physics professors taught us to think of tensors as "arrays with units." If it's a vector/matrix/higher dimensional array but has physical units, it's probably a tensor. The fact that it has units means it represents something ph…

This is a pretty tired line, I gotta say.

Obviously they are not. One is a linear operator, the other is a data structure for implementing computations using that operator. This description extends to all tensors.

It's like saying "queues are not just lists". That is true and also neither insightful nor helpful.

I don't see it as mystifying or complicated, what am I missing?

Post reply on HN