71–80 of 95 posts
What is some good intuition around matrix inverses?
There are some other interpretations, for example if you think of a linear system Ax=b as a kind of “checking machine” (does x satisfy Ax=b? Yes/no, try again), then the inverse of A can produce solutions, since x=(A inverse)b.
Earlier quoted context omitted.
Purist test: aren't vectors just Nx1 matrices? Joking aside, matrices are linear maps in the context of multiplication. They can absolutely have meanings assigned outside the usual "when multiplied by a vector" type of calculations. For example the "adjacency matrix" representation of a graph is very often never multiplied by a vector even in academic algorithm descriptions. YMMV if you call it a (lookup) table in th…
> aren't vectors just Nx1 matrices? It's been a while since I formally studied maths, but - can you describe a situation in which this is _not_ true? (modulo transposition)
Similarly, in a vector space there can be abstract vectors, which exist no matter of what basis you choose to write them down. (Choosing a basis = choosing a way to encode them as a 1xN or Nx1 matrix).
Often in the theory, general theorems can be proven very efficiently using these abstract vectors, but most of the time when one wants to actually do things to those vectors, picking a convenient basis and working with N coordinates is the best choice.
If we're talking about 3D rendering then a matrix multiplication with a vector are just projections: A 3x3 matrix that transforms a vector is nothing else than that vector being projected onto the 3 axis which are inside the 3x3 matrix. This is very easy to see visually. A matrix times matrix multiplication (eg 3x3 times 3x3) is just projecting the axis of one matrix onto the other: expressing the coordinate system i…
Earlier quoted context omitted.
> Purist test The short answer is. Yes, but if your PL treats it that way you will wind up in trouble [0]. The long answer is: https://www.youtube.com/watch?v=C2RO34b_oPM (but seriously take the time to watch this, it's fantastic). [0] Separately C- and often C-derived languages/frameworks like numpy also treat them as 1xN matrices, but that's a nearly tabs-vs-spaces level religious war, with actual consequences. I t…
> numpy also treat them [vectors] as 1xN matrices I may be misunderstanding something, but in numpy at least a vector, a 1×N array ("row vector"), and a N×1 array ("column vector") behave differently. vec = numpy.array([1,2,3]) assert vec.shape == (3,) assert vec.ndim == 1 row = np.array([[1,2,3]]) assert row.shape == (1,3) assert row.ndim == 2 col = np.array([[1], [2], [3]]) assert col.shape == (3,1) assert col.ndim…
>>> import numpy as np
>>> vec = np.array([1, 2, 3])
>>> mtx = np.eye(3)
>>> np.dot(vec, mtx)
array([1., 2., 3.])
>>> np.dot(mtx, vec)
array([1., 2., 3.])
I'd prefer stating that matrices actually don't mean anything in isolation. They are simply representations of linear maps. What happens to the elements of a matrix before the elements are formed is irrelevant to the matrix itself. ..And to be clear, I am not a purist.
Purist test: aren't vectors just Nx1 matrices? Joking aside, matrices are linear maps in the context of multiplication. They can absolutely have meanings assigned outside the usual "when multiplied by a vector" type of calculations. For example the "adjacency matrix" representation of a graph is very often never multiplied by a vector even in academic algorithm descriptions. YMMV if you call it a (lookup) table in th…
Earlier quoted context omitted.
That is genuinely a really fun way to look at it, thank you for linking this! Especially this fits nicely with Markov matrices where you have N input nodes and N output nodes and the sum of all of the probabilities coming out of one of the nodes needs to equal 1. What I might find a little more difficult to teach to people through this lens is the phenomenon of eigenvectors, but I suppose that's to be expected—nothin…
>What I might find a little more difficult to teach to people through this lens is the phenomenon of eigenvectors Why? Eigenvectors are simply inputs to the network where the output keeps its shape, that is, at most it gets rescaled, as if you had applied a uniform gain to the components, but otherwise it will be the same as the input.
- does a matrix have the same left-eigenvectors as its right-eigenvectors?
- what is the relationship between the left-eigenvalues and right-eigenvalues?
- is there always an eigenvector? when is there a complete set? how do you generalize your notion of eigenvectors so that matrices always have a complete set of them?
I am not sure any of these are intuitive here.
Earlier quoted context omitted.
>What I might find a little more difficult to teach to people through this lens is the phenomenon of eigenvectors Why? Eigenvectors are simply inputs to the network where the output keeps its shape, that is, at most it gets rescaled, as if you had applied a uniform gain to the components, but otherwise it will be the same as the input.
So for example let me ask for your knee-jerk opinions based on this idea: - does a matrix have the same left-eigenvectors as its right-eigenvectors? - what is the relationship between the left-eigenvalues and right-eigenvalues? - is there always an eigenvector? when is there a complete set? how do you generalize your notion of eigenvectors so that matrices always have a complete set of them? I am not sure any of thes…
Notice that u and v cannot be equal, because they are not the same shape. However, if v is a right eigenvector with eigenvalue \lambda, then v^T is a left eigenvector with eigenvalue \lambda, as well. More or less what this means is that we tend to just ignore left eigenvalues and only use the right eigenvalues, because the math is exactly the same up to a transpose operation.
A matrix does not necessarily have nontrivial eigenvectors. Think about the 0 matrix here. But, if A is nonzero, then it must have at least one nonzero eigenvalue, hence one nontrivial eigenvector. This is because a nonzero matrix must have at least one nonzero row and column; however, if you construct the other rows (columns) to be multiples of the first column, you end up with a matrix with only one nontrivial eigenvalue. This also illustrates the conditions necessary for an nxn matrix A to have n linearly independent eigenvectors: the rows of A must be linearly independent.
All of this is covered in a decent undergrad linear algebra course. I would suggest either finding a video course, or getting a good book and working through it, if you want to understand these things better.
Earlier quoted context omitted.
> numpy also treat them [vectors] as 1xN matrices I may be misunderstanding something, but in numpy at least a vector, a 1×N array ("row vector"), and a N×1 array ("column vector") behave differently. vec = numpy.array([1,2,3]) assert vec.shape == (3,) assert vec.ndim == 1 row = np.array([[1,2,3]]) assert row.shape == (1,3) assert row.ndim == 2 col = np.array([[1], [2], [3]]) assert col.shape == (3,1) assert col.ndim…
I just tried this, I stand corrected, but I remember this (the last clause) not working in numpy. It's possible my memory is flawed: >>> import numpy as np >>> vec = np.array([1, 2, 3]) >>> mtx = np.eye(3) >>> np.dot(vec, mtx) array([1., 2., 3.]) >>> np.dot(mtx, vec) array([1., 2., 3.]) It's possible though that I was remembering not that it doesn't execute, but rather that if you do mess up the multiplication order…
points = np.array([[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0]])
M = np.array([[0.866, 0.5, 0], [-0.5, 0.866, 0], [0, 0, 1]]) # rotation
points @ M # works
M @ points # raises ValueError, also a different transform
However, I think you can track dimensional compatibility using xarray:
import xarray as xr
import numpy as np
points = xr.DataArray([[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0]], dims=("idx", "a"))
M = xr.DataArray([[0.154, 0.988, 0], [-0.988, 0.154, 0], [0, 0, 1]], dims=("a", "b")) # rotation
points.dot(M) # works
M.dot(points) # also works
# M's a-dim is matched to point's a-dim, so both
# results have same values regardless of
# multiplication order, just transposed.
assert np.all(points.dot(M).T == points.dot(M))