Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fundamentals of Linear Algebra and Optimization [pdf]

seas.upenn.edu

Post reply on HN
Print view View original

31–40 of 63 posts

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#31

Post by jacobolus »

catnaroek wrote: post #27

Did fields other than R and C eat the authors' children or something?

Often linear algebra tools are applied to Hilbert spaces or Banach spaces, where fields other than R or C don't make much sense.

Looking at page 25 of TFA:

> In Definition 1.2, the field R may be replaced by the field of complex numbers C, in which case we have a complex vector space. It is even possible to replace R by the field of rational numbers Q or by any other field K (for example Z/pZ, where p is a prime number), in which case we have a K-vector space (in (V3), ∗ denotes multiplication in the field K). In most cases, the field K will be the field R of reals.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#32

Post by ivansavz »

spraak wrote: post #24
SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I'm curious what you think of the "No Bullshit Guide to Linear Algebra" [1]? I'm considering buying it to refresh my knowledge from school. Or what books do you suggest? [1] https://www.amazon.com/No-bullshit-guide-linear-algebra/dp/0...

Here is a direct link to the page in the preview of the book where the matrix-product-is-the-same-as-linear-transformation idea first appears: https://minireference.com/static/excerpts/noBSguide2LA_previ...

I can't quite tell if the book passes the test, since the entry-point is the definition of matrix-vector product which is very close to a "complicated formula with a couple of nested sigmas" but the book also mentions the notion of matrix representations, so hard to tell overall.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#33

Post by catnaroek »

jacobolus wrote: post #31
catnaroek wrote: post #27

Did fields other than R and C eat the authors' children or something?

Often linear algebra tools are applied to Hilbert spaces or Banach spaces, where fields other than R or C don't make much sense. Looking at page 25 of TFA: > In Definition 1.2, the field R may be replaced by the field of complex numbers C, in which case we have a complex vector space. It is even possible to replace R by the field of rational numbers Q or by any other field K (for example Z/pZ, where p is a prime numb…

Yeah, lots of courses pay lip service to the existence of fields of characteristic p during the initial few weeks, then proceed to ignore them completely.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#34

Post by kccqzy »

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I don’t necessarily agree though. I’ve seen books first trying to introduce linear transformation without matrices and frankly to a beginner it’s even more confusing to be told f is linear if f(ax+y)=af(x)+f(y). Some students really need a concrete grasp of matrices with actual numbers they are familiar with before going into the abstract definition.

I’ve seen worse though: those that attempt to shoehorn abstract algebra in the process by first rigorously defining a field.

And no, teaching multiplication doesn’t involve memorizing a formula; it’s a simple mechanical process of arranging one matrix on the left the other at the top and multiplying/adding their corresponding rows and columns. Once this process is familiar to a student, they will have no trouble writing out the formula with nested sigmas.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#35

Post by closeparen »

hvs wrote: post #13

I may be a troglodyte, but I was bored senseless in my linear algebra, calculus, discrete mathematics, statistics, etc classes in college. Then they tag all the interesting stuff like AI (we didn't call it Machine Learning back in the 90s) at the end of your major where you actually use it. I had to go back and relearn it all because I didn't pay attention the first time. Note to mathematicians (or at least math prof…

Same: I never gave a crap about algebra, or calculus, until AP physics. When I learned that the point (and the origin story!) of all this is to model the behavior of the universe I suddenly wished I had been paying attention for years prior.

Seconding your wish that curricula would lead with motivations and then drill mechanics rather than drilling mechanics for 11 years and finally giving you the motivation in year 12.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#36

Post by jknz »

SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I would teach linear algebra with a book that barely mentions linear transformations.

In finite dimensions, linear transformations and matrices are exactly the same object mathematical objects, with very different notations (matrix notation (boxes with numbers inside) vs the linear space/linear transformation notation). I would rather the students to learn deeper mathematics only in matrix notation, rather than to master less substantial mathematics with both notations.

Teaching both notations may reinforce the idea that matrices and linear transformations are different mathematical objects in finite dimension. Teaching the more abstract notation is mainly useful in infinite dimension (Hilbert spaces).

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#37

Post by kccqzy »

nextos wrote: post #29
SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

Do you have any other tests for calculus, analysis, abstract algebra, discrete mathematics, probability or statistics books?

For me my test for multivariable calculus text is their treatment of the chain rule. If the book says something like “derivative of a composition is the composition of the derivatives” it’s good. If they instead say something involving a sigma and things like ∂f/∂x ∂x/∂t it’s likely bad.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#38

Post by nafizh »

zengid wrote: post #4
alexcnwy wrote: post #2

This is great! There's so much hype about machine learning but so few people seem to appreciate the fundamental importance of linear algebra (especially in ML). This fast.ai course is also great: http://www.fast.ai/2017/07/17/num-lin-alg/

Cool new course! I'm thinking about taking their Practical Deep Learing for Coders course at the moment: http://course.fast.ai/

Wait till January when they will release version 2 of this course which teaches DL with pytorch. I was one of the international fellows for that course, and it is so good you would want to do the updated one.

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#39

Post by linkmotif »

begriffs wrote: post #20
SonOfLilit wrote: post #15

My test for linear algebra books is how they first present matrices and matrix multiplication. If they define a matrix as an NxM table of numbers with a multiplication operation defined as this complicated formula with a couple of nested sigmas, and then much later a lemma is mentioned that says every linear transformation can be represented as a matrix and then the composition of two transforms is the matrix multipl…

I think the book you're talking about is Axler's "Linear Algebra Done Right." I worked through the problems in this book with some friends, and it is a very good presentation. After doing that book I wanted to see what other books were out there, which ones were the good ones, and sort of classify them. I came up with this list: https://begriffs.com/posts/2016-07-24-best-linear-algebra-bo... I classify the books as G…

Aweomse. Thank you. Jealous of you and your friends. Wish I had a Pythagorean squad :)

Re: Fundamentals of Linear Algebra and Optimization [pdf]

#40

Post by colmvp »

hvs wrote: post #13

I may be a troglodyte, but I was bored senseless in my linear algebra, calculus, discrete mathematics, statistics, etc classes in college. Then they tag all the interesting stuff like AI (we didn't call it Machine Learning back in the 90s) at the end of your major where you actually use it. I had to go back and relearn it all because I didn't pay attention the first time. Note to mathematicians (or at least math prof…

It reminds me a bit of the motivation behind fast.ai, that you should get your hands dirty with the things you care about instead of learning the tough things until you're ready.

I found it easier to learn things like stats, linear algebra, and calculus once I had a personal application to it. Even something that I love in Reinforcement Learning, I would sometimes get sleepy reading pages and pages from the Sutton and Barto book, but as soon as I worked on a coding exercise, I could spend hours actively participate with trying to solve the problem.

Post reply on HN