Fundamentals of Linear Algebra and Optimization [pdf] • Hacker News Board

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#51

Post by wolfgke » Fri, Dec 15, 2017, 1:13 PM UTC

jknz wrote: post #36

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I would teach linear algebra with a book that barely mentions linear transformations. In finite dimensions, linear transformations and matrices are exactly the same object mathematical objects, with very different notations (matrix notation (boxes with numbers inside) vs the linear space/linear transformation notation). I would rather the students to learn deeper mathematics only in matrix notation, rather than to ma…

> In finite dimensions, linear transformations and matrices are exactly the same object mathematical objects

They are not: Matrices represent linear transformations with respect to a given basis. Linear transformations are completely independent from any chosen basis.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#52

Post by SonOfLilit » Fri, Dec 15, 2017, 2:45 PM UTC

spraak wrote: post #24

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I'm curious what you think of the "No Bullshit Guide to Linear Algebra" [1]? I'm considering buying it to refresh my knowledge from school. Or what books do you suggest? [1] https://www.amazon.com/No-bullshit-guide-linear-algebra/dp/0...

It... more or less passes the test?

I mean, it defines matrix-vector product in a nice but abstract way, and then in the next paragraph explains why we chose that definition. And it does say "this, this here is the one important idea in this book", which it gets many points for.

I really would prefer a textbook to start with "ok, here's something we want to do. Lets figure out a formula for it. Now lets give it the name matrix-vector product".

Same for matrix-matrix products.

I don't know any really good linear algebra books, in my school it was taught the worst possible way (define a field, rote-learn the mechanical operation of matrix multiplication, talk about vector spaces for a while, talk about matrices, first mention linear transformations). Some people on this thread gave other suggestions, I'd start reading a few and choose the one I connect with the best.

Anyway, I really learned linear algebra from using it.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#53

Post by wolfgke » Fri, Dec 15, 2017, 3:43 PM UTC

hvs wrote: post #13

I may be a troglodyte, but I was bored senseless in my linear algebra, calculus, discrete mathematics, statistics, etc classes in college. Then they tag all the interesting stuff like AI (we didn't call it Machine Learning back in the 90s) at the end of your major where you actually use it. I had to go back and relearn it all because I didn't pay attention the first time. Note to mathematicians (or at least math prof…

> lead with the interesting stuff before the theory. It'll make learning easier and more fun.

As a mathematician I say:

What is considered as interesting is different for each person. I, personally, for example deeply love this really abstract stuff (but of course I am aware that other people have different preferences). So I would say even finding "interesting stuff" that many students in the lecture hall might be interested in is really, really hard.

Another important argument against your idea is: To be even able to formulate the ideas from AI, one first have to learn and understand the words of the language in which one will formulate this. And these words are like "vector space", "linear map", "tensor product" etc. and understanding their meaning means knowing theorems about them. Starting with advanced topics, such as your AI example, is like giving beginner language learners a really advanced text in the foreign language that they just begin to learn. In other words: A reallz dubious idea.

Considerung your post I can only ask you why you did not have a talk with the course advisor of your faculty. He would immediately have told you why these lectures are important for the things that you are actually interested in.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#54

Post by internetman55 » Fri, Dec 15, 2017, 7:47 PM UTC

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I don't understand what's so wrong about that. Surely the interested student would like to learn about the properties of matrix arithmetic as well as vector spaces, linear transformations and their representation as matrices, etc.

I've had a quite accomplished mathematician for a professor who was very adamant that students should learn about basic matrix operations before the abstract theory. According to him, his undergraduate education at Princeton did things the "right" way, but left him quite confused about the main ideas and motivations. He ended up making some quite important contributions to the theory of representations of some certain algebraic structures if I recall correctly.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#55

Post by internetman55 » Fri, Dec 15, 2017, 7:50 PM UTC

tomjakubowski wrote: post #23

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

Strang’s textbook (and lectures) introduce matrix multiplication AB as combinations of A’s columns. He uses the cell-by-cell / “dot product” formulas just to check his work, which seems reasonable to me. I’m a big fan of “combinations of columns”. I sometimes even hear that in Strang’s voice when I’m doing some matrix multiplications. Axler was way too abstract for me as a first linear algebra book. I look forward to…

Strang's exposition can make you a wizard of matrix multiplication and give you quite a good intuition for various properties of matrix operations, in my opinion.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#56

Post by sn9 » Sat, Dec 16, 2017, 5:22 AM UTC

begriffs wrote: post #20

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I think the book you're talking about is Axler's "Linear Algebra Done Right." I worked through the problems in this book with some friends, and it is a very good presentation. After doing that book I wanted to see what other books were out there, which ones were the good ones, and sort of classify them. I came up with this list: https://begriffs.com/posts/2016-07-24-best-linear-algebra-bo... I classify the books as G…

I've heard Shilov's text described as a great complement to Axler's.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#57

Post by igniousrock » Sat, Dec 16, 2017, 10:14 PM UTC

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

To me, matrices and matrix multiplication really represent the "calculus" of linear algebra. When you are studying real analysis, the theory part (closed, open sets, definition of limit points, continuity, etc) are not actually that helpful when doing a computation (calculating and integral or derivative). It often makes sense to teach a student the calculation side before the theory because its easier to get a intuition for.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#58

Post by SonOfLilit » Sun, Dec 17, 2017, 6:07 AM UTC

internetman55 wrote: post #54

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I don't understand what's so wrong about that. Surely the interested student would like to learn about the properties of matrix arithmetic as well as vector spaces, linear transformations and their representation as matrices, etc. I've had a quite accomplished mathematician for a professor who was very adamant that students should learn about basic matrix operations before the abstract theory. According to him, his u…

They way I was taught was like learning how to do long-form multiplication without mentioning that "the product of 5 and 6 is 5+5+5... 6 times".

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#59

Post by Tomte » Sun, Dec 17, 2017, 11:44 AM UTC

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

Stupid question: is the order of elements in matrix notation just convention or is there a deeper reason for it? Could we just as well have used „transposed matrices“?

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#60

Post by apricot » Sun, Dec 17, 2017, 5:06 PM UTC

kccqzy wrote: post #37

nextos wrote: post #29

Earlier quoted context omitted.

Do you have any other tests for calculus, analysis, abstract algebra, discrete mathematics, probability or statistics books?

For me my test for multivariable calculus text is their treatment of the chain rule. If the book says something like “derivative of a composition is the composition of the derivatives” it’s good. If they instead say something involving a sigma and things like ∂f/∂x ∂x/∂t it’s likely bad.

What do you mean by "derivative of a composition is the composition of the derivatives"?