Live data from Hacker News

Visualizing matrix multiplication as a linear combination

eli.thegreenplace.net

21–30 of 30 posts

Re: Visualizing matrix multiplication as a linear combination

#21
Here's how you can apply this interpretation of matrix multiplication:

Think of the columns of the matrix A as basis vectors of a coordinate system represented in global coordinates, and the vector v as the components of a vector in that coordinate system. Then the product A * v transforms the vector v into the global coordinate system.

Carrying this forward, matrix multiplication A * B gives the combined representation of two coordinate transformations.

Re: Visualizing matrix multiplication as a linear combination

#22
post #7
post #4

Hmm. As someone who has never understood/visualized/been taught matrix multiplication in any other way than how it's shown on the linked page, could someone explain the alternative? I.e., if this is new, what mental model do you currently have? Presumably that one would be new and useful for me. Edit: "Typically this visualization isn't taught" -- that's what I"m asking. What visualization were you taught, if not thi…

Typically, this visualization isn't taught, in my experience. What's taught is the formula for computing each cell of the result matrix (cell i,j being the dot product of row i of the first matrix with column j of the second). While this is, of course correct, and also the most efficient way to compute the multiplication manually, it's not always clear why the formula is correct. Hopefully the visualization on the li…

Really? Interesting. We spent some time discussing multiplication by block decompositions in my 100-level linear algebra class -- is this really uncommon?

Re: Visualizing matrix multiplication as a linear combination

#23
post #21

Here's how you can apply this interpretation of matrix multiplication: Think of the columns of the matrix A as basis vectors of a coordinate system represented in global coordinates, and the vector v as the components of a vector in that coordinate system. Then the product A * v transforms the vector v into the global coordinate system. Carrying this forward, matrix multiplication A * B gives the combined representat…

Another way to look at it: A matrix is a linear transformation, and multiplying a vector by a matrix is how you apply the transformation. But linear transformations are really just changes of basis. How do you change your basis? You find the dot product of a vector with each new basis vector. And that's exactly what matrix multiplication is. When you multiply your column vector by a row in the matrix, you're finding the dot product, doing the projection in your change of basis.

Re: Visualizing matrix multiplication as a linear combination

#24
post #21

Here's how you can apply this interpretation of matrix multiplication: Think of the columns of the matrix A as basis vectors of a coordinate system represented in global coordinates, and the vector v as the components of a vector in that coordinate system. Then the product A * v transforms the vector v into the global coordinate system. Carrying this forward, matrix multiplication A * B gives the combined representat…

[deleted]

Re: Visualizing matrix multiplication as a linear combination

#25
There are 2 types of matrix multiplications which should not be confused with each other.

The first type is a change of basis where the columns of the matrix are the old basis represented in the new basis.

The second type is a linear function that takes in a vector and outputs another vector.

Re: Visualizing matrix multiplication as a linear combination

#27
post #14

If you want more, I can definitely recommend the video lectures from MIT's linear algebra course by Prof Gilbert Strang: http://web.mit.edu/18.06/www/videos.shtml

I concur on these videos, everybody who teaches the subject should watch him. You get a lot of "aha! it's that simple?" moments watching him.

Re: Visualizing matrix multiplication as a linear combination

#28
Sure, in the matrix product

AB = C,

C has the same number of rows as A and the same number of columns as B, and each column j of C is from column j of B acting as coefficients in a linear combination of all the columns of A.

Next: For the set of real numbers R, positive integers m and n, m by n real matrix A, real numbers a and b, and n by 1 x and y, we have that

A(ax + by) = (aA)x + (bA)y

so that A is linear.

If we regard x and y as vectors in the n-dimensional real vector space R^n, then we have that A is a function

A: R^n --> R^m

and a linear operator which can be good to know.

Then the subset of R^n

K = { x | x in R^n and Ax = 0 }

(where 0 is the m by 1 matrix of all zeros) is important to understand. E.g., K = [0] if and only if function A is 1-1. If m = n, then A has an inverse A^(-1) if and only if A is 1-1.

With C = AB, the matrix product is the same as function composition so that

Cx = (AB)x = A(Bx)

which also can be good to know and, of course, uses just the associative law of matrix multiplication.

That matrix multiplication is associative is a biggie -- sometimes says some big things, e.g., is the core of duality theory in linear programming.

And the situation is entirely similar for the complex numbers C in place of the real numbers R.

Re: Visualizing matrix multiplication as a linear combination

#29
post #14

If you want more, I can definitely recommend the video lectures from MIT's linear algebra course by Prof Gilbert Strang: http://web.mit.edu/18.06/www/videos.shtml

+1 for the Strang videos. He's an amazing lecturer. e.g. in Lecture 1 explains very clearly matrix multiplication in the "column picture" and the "row picture". More intuition than you can shake a stick at: http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-...

This tutorial is also pretty good for geometrical intuition: http://www.cns.nyu.edu/~eero/NOTES/geomLinAlg.pdf

Re: Visualizing matrix multiplication as a linear combination

#30
post #3

This article was posted a few weeks ago, helped me understand matrix operations I've been trying to wrap my head around for years: http://betterexplained.com/articles/linear-algebra-guide/

I'm actually studying linear algebra right now, and this site has given me insights my text book (which is actually pretty good!) just hasn't. Thanks for the link :-)
Post reply on HN