> "For an avid student with great skill in mathematics, rushing through the standard curriculum is not the best answer. That student who breezed unchallenged through algebra, geometry, and trigonometry, will breeze through calculus, too." That was me. I was great at calculus type things, but Matrix Theory hit me like a ton of bricks. I still have that text book, sitting on my other desk, staring menacingly at me from…
First, Horn and Johnson is a bit much. I was in Horn's class. I had done a LOT in, call it, linear algebra and matrix theory in my career before the class, told the profs I didn't need the course, and they said it was a "second, advanced course" and smiled.
The course was quite competitive and without trying at all and without intending to be competitive, I effortlessly blew away all the other students on graded homework, the tests, the midterm, the final exam, and the corresponding qualifying exam. At the end of the course Horn wrote about me IIRC "Best performance in the class by a wide margin. Knows this material cold."
So, yes, it was an advanced course, actually had a lot of nice stuff in it, Horn's lectures were nicely precise and at times with some unusual, nice approaches, but to do well in the course it was sufficient just to have had a good background before.
What background? For the main books, E. Nering (a student of E. Artin at Princeton), Halmos (an assistant to von Neumann at the Institute of Advanced Study at Princeton), Finite Dimensional Vector Spaces, basically also a finite introduction to Hilbert space and the spectral theorem there, Forsythe and Moler, Computer Solutions of Linear Algebraic Systems, and some good texts in multivariate statistics with regression analysis, discriminate analysis, factor analysis, analysis of variance. More in applications, e.g., the fast Fourier transform, more on curve fitting, linear systems in electronic engineering, antenna theory and beam forming, optimization, linear programming, unconstrained optimization, the Markowitz and Sharpe applications to finance, Lagrange multipliers, the Kuhn-Tucker conditions, etc. can also help.
But Horn is not a good choice for a first text. For a first or second text I'd suggest, say, Hoffman and Kunze, Linear Algebra, Second Edition available for free on the Internet.
For more, see my post on math in
https://news.ycombinator.com/item?id=15116379
and there sections
(2) Linear Algebra
(2.1) Linear Equations
(2.2) Gauss Elimination
(2.3) Vectors and Matrices
(2.4) Vector Spaces
(2.5) Eigen Values, Vectors
(2.6) Texts
To be brief, about the earliest and easiest start on linear algebra and matrix theory is just a high school style system of linear equations. The main solution technique is Gauss elimination. Matrix notation is a better notation for that subject.
Here is essentially the role of matrix theory: Each of the old results in linear algebra can be written as a result, with nicer notation, in matrix theory. Can get the same results without matrix notation, but matrix notation makes it all much easier.
Next, a broad statement is that the two pillars of the field of analysis in math are (1) linearity and (2) continuity. Well, linear algebra and matrix theory stands strongly on linearity and, as we move on in both the theory and applications, also continuity.
Let's be clear on linearity via linear algebra and matrix theory: So, for positive integers m and n and an m x n matrix A we say that matrix A is a linear transformation (function) if for all n x 1 vectors x and y, and numbers a and b, we have that
A(ax + by) = aAx + bAy
Sure, to read this need the definitions of matrix sum and product; sum is trivial; product is not much harder and is really just what need to make Ax = b be the same as the high school system of linear equations.
For the numbers, usually use either the set of real numbers R or the set of complex numbers C. But, sure, for numerical computation are essentially limited to the set of rational numbers Q. But in general need only what a course in abstract algebra calls a field: Each of R, C, and Q is such a field but also the set of integers modulo a prime number is a field, of interest in algebraic coding theory and cryptology.
This definition of linearity generalizes in Hilbert space, Banach space, and functional analysis, and the more general definitions and results are important in quantum mechanics, differential equations in science and engineering, signal processing in electronic engineering, etc. Again, linearity is a pillar of analysis in math.
Why pillars? In both theory and applications, linearity and continuity commonly hold and are astoundingly powerful properties. For such applications we have multivariate statistics, optimization, electronic engineering, antenna theory, beam forming, signals (each time invariant linear system has sines and cosines as eigenvectors; when a violinist on a concert stage plays some pure tones, the concert hall transmits those tones to you in the audience as a linear system so that what you hear are just the pure tones with the right frequencies but with some phase and amplitude changes; the Navy likes to know that for sonar signals; the USAF likes to know that for radar signals; cell phone people like to know that for their signals), Fourier theory, linear partial differential equations, superposition in quantum mechanics, etc. And when linearity does not hold, commonly it is a good, first approximation and the main means of iterative techniques. And if a problem is not linear, maybe after some simple transformation it will be.
In some of the posts here, there is mention of matrix theory and basis, that is, a coordinate system. Well, can do that although is it not nearly as general as what physics likes to do with coordinate systems. But also can just decide not to do that, to take the vector space as just the n-tuples and not force thinking of the n-tuples as just coordinates of vectors in some basis. Or can do either approach depending on what is easier in the context.
Here is a point should get: Suppose we start with just systems of linear equations. Then we say we are working with n-tuples of numbers. Then we call those n-tuples a vector space. Then using essentially just the main, relevant properties of those n-tuples, we write down the definition, axioms, of a vector space where we've said nothing about the vectors but have left them as just points.
Well then we have two advantages: First, the definition of a vector space lets us talk about subspaces and in particular subspaces of the vector spaces of just the n-tuples, and we want to do that already, strongly with just Gauss elimination for linear equations. E.g., with the linear system Ax = b where m, n are positive integers, A is m x n, x is n x 1, and b is m x 1, the set of all x so that Ax = 0 (m x 1 of all zeros) is a vector subspace of all the n x 1 vectors (for the set of real numbers R, commonly called the set R^n). Call the set K the set of all x so that Ax = 0. If for some n x 1 u we have that Au = b, then from linearity we can argue that any v so that Av = b can be written as a sum of u and some vector in K. In this way we see all possible solutions of Ax = b. Actually at the end of Gauss elimination we can see K and u easily enough.
Second, we get to consider vectors other than just n-tuples. E.g., we can consider the data of 1 second of music as a vector, a random variable as a vector, a color as a vector, the wave function of a photon or electron as a vector, etc.
Then as the book continues, we get into eigenvalues and eigenvectors. Eigen is German essentially for special. They are special, and valuable. Maybe the nicest part is the polar decomposition: Each square matrix is a product UH where U in unitary and H is Hermitian. In class, when Horn got to that, I shouted out "That's my favorite theorem! The unitary part is an isometry, that is, doesn't change lengths or angles and is essentially a rigid motion, maybe just a rotation or reflection. The Hermitian part H is a shocking dream, amazing beyond belief: All H can do is take a circle and make it into an ellipse: The two axes of the ellipse are perpendicular (orthogonal) and the eigenvectors. Their lengths are the eigenvalues. And this generalizes to rounded footballs in three dimensions and all finite dimensions. And with the spectral theorem it generalizes to infinitely many dimensions and is the main reason in quantum mechanics the observables are eigenvalues. The polar decomposition is also the source of the powerful singular value decomposition, principle components analysis, factor analysis, analysis of saddle points in optimization (see W. Fleming, Functions of Several Variables), the matrix condition number in the numerical analysis of Gauss elimination, and much more in theory and applications.
Hope this helps.