Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Intuitive Guide to Linear Algebra (2012)

betterexplained.com

Post reply on HN
Print view View original

51–60 of 105 posts

Re: An Intuitive Guide to Linear Algebra (2012)

#51

Post by UweSchmidt »

sushisource wrote: post #3

I love explainers like this, but it frankly makes me a little angry that the vast majority of the math teachers I had in highschool and college taught in the awful way described in the setup to the piece. Why is that? Has anyone studied it, or is there even a solid anecdotal explanation? The best one I can imagine is many of these professors simply don't care much for teaching and are more focused on their research,…

I had an eye opening experience and some closure about my own eduction after reading a certain book, a collection of short biographic accounts of famous Germans and their horrific experience in school during the last few centuries. School has always been bad and actually used to be borderline insane, and yet these great characters, politicians, artists and scientists still developed fine.

Rant: If society gave a shit teacher would be the highest paid job with the highest standards, both in knowledge and in teaching skill. I mean literally 500k/year for the most noble calling on earth. The kind of progress humanity would make with a generation taught by the best, and teaching itself revolutionized, would be crazy.

https://www.amazon.de/Deutsche-Schulzeit-Erinnerungen-Erz%C3...

Re: An Intuitive Guide to Linear Algebra (2012)

#52

Post by quietbritishjim »

nightcracker wrote: post #24
vecter wrote: post #12

This is ok but nothing is as intuitive as 3B1B's series on YouTube that has been posted hundreds of times on HN [0]. Linear algebra is really about linear transformations of vector spaces, which is not captured in this blog post. [0] https://www.youtube.com/watch?v=fNk_zzaMoSs

> Linear algebra is really about linear transformations of vector spaces, which is not captured in this blog post. I... disagree. Some of linear algebra is about that. And it's probably a good way to view it that way when learning. But some of my current work (coding theory) involves linear algebra over finite fields. We use results from linear algebra, and interpret our problem using matrices, but really at no point…

> > Linear algebra is really about linear transformations of vector spaces, which is not captured in this blog post.

> I... disagree.

This is literally the definition of the term "linear algebra".

> really at no point are we viewing what we're doing as transforming a vector space, we're just solving equations with unknowns.

You may not see what you're doing as transforming vector spaces with linear operators, but that is what you're doing. It's worth pointing out that the definition of vector spaces allows any field, including finite ones, though it's true that the intuition won't be exactly the same.

Another way to say this: if you're working on a problem without thinking about the connection to linear transformations, then it's not correct to say it's a linear algebra problem without obvious connection to linear transformations; instead, it's not a linear algebra problem at all, by definition.

Re: An Intuitive Guide to Linear Algebra (2012)

#53

Post by smallnamespace »

xscott wrote: post #26
nightcracker wrote: post #24

Earlier quoted context omitted.

> Linear algebra is really about linear transformations of vector spaces, which is not captured in this blog post. I... disagree. Some of linear algebra is about that. And it's probably a good way to view it that way when learning. But some of my current work (coding theory) involves linear algebra over finite fields. We use results from linear algebra, and interpret our problem using matrices, but really at no point…

I think this is spot on. Depending on what you're doing, a matrix can be: - A linear transformation - A basis set of column vectors - A set of equations (rows) to be solved - (your example: parity equations for coding theory) - The covariance of elements in a vector space - The Hessian of a function for numerical optimization - The adjacency representation of a graph - Just a 2D image (compression algorithms) ... (I'…

A covariance matrix naturally transforms from the measured space to a space where things are approximately unit Gaussian distributed. This is identical to the Z transform in 1D case.

This can be useful in, say, exotic options trading - a natural unit of measurement is how many ‘vols’ an underlier has moved, e.g. a 10-vol move is very large.

Re: An Intuitive Guide to Linear Algebra (2012)

#54

Post by obastani »

xscott wrote: post #26
nightcracker wrote: post #24

Earlier quoted context omitted.

> Linear algebra is really about linear transformations of vector spaces, which is not captured in this blog post. I... disagree. Some of linear algebra is about that. And it's probably a good way to view it that way when learning. But some of my current work (coding theory) involves linear algebra over finite fields. We use results from linear algebra, and interpret our problem using matrices, but really at no point…

I think this is spot on. Depending on what you're doing, a matrix can be: - A linear transformation - A basis set of column vectors - A set of equations (rows) to be solved - (your example: parity equations for coding theory) - The covariance of elements in a vector space - The Hessian of a function for numerical optimization - The adjacency representation of a graph - Just a 2D image (compression algorithms) ... (I'…

The first three can reasonably be thought of as defining linear transformations. For linear systems of equations A x = b, x is an unknown vector in the input space that is mapped by A to b.

Both covariance matrices and Hessians are more naturally thought of as tensors, not matrices (and therefore not linear transformations). That is, they take in two vectors as input and produce a single real number as output.

As for graph adjacency matrix, this can actually be thought of as a linear transformation on the vector space where the basis vectors correspond to nodes in the graph. Linear combinations of these basis vectors correspond to probability distributions over the graph (if properly normalized).

2D images... Yes, these cannot really be interpreted as linear transformations. But I'd say these aren't really matrices in the mathematical sense.

Re: An Intuitive Guide to Linear Algebra (2012)

#55

Post by adamcharnock »

greentimer wrote: post #50
adamcharnock wrote: post #36

I... I still really struggle with this. I'm a smart person, I've got a bachelors of engineering, I've been a professional software developer for around 14 years now, and I've built a house. But there is something about degree-level maths and beyond that I find deeply unintuitive in a way that software development isn't. Through comments here I found 3blue1brown's (clearly much loved) videos. By the third video I was…

Higher math involves a very different way of thinking from the typical, useful things people do for a living. Exactness is important. The abstractions can run very deep. It's easy to get lost in the pure side of things without really understanding how to apply it.

I think this is possibly the crux of it for me. I've certainly got well developed abstract thinking for software development, but software has always had a clear application for me, so the abstract thinking developed as a matter of course.

I've rarely found any higher math instruction which takes for the form, "so you have this specific problem X, here is how we can solve it with technique Y"[1]. But I suspect that it is because it is higher math (which presumably means 'higher order' math).

Without this, and without an inherent enjoyment of the pureness of the math, it seems somewhat esoteric for me personally. I'm not complaining, nor do I really think it should be any other way. I'm just reflecting on it really.

This also makes me think of my foray into monads: "The thing about monads is once you finally understand them you immediately lose the ability to explain what they are to others." Not saying that's the case here, just feels related.

[1] At least where I found problem X to be satisfactory. I didn't find my lecturer's problem of, "you're stood on a mountain described by this PDE, on what vector must one walk in order to stay at the same altitude" to be very applicable. I was a pretty wilful student though.

Re: An Intuitive Guide to Linear Algebra (2012)

#56

Post by xscott »

obastani wrote: post #54
xscott wrote: post #26

Earlier quoted context omitted.

I think this is spot on. Depending on what you're doing, a matrix can be: - A linear transformation - A basis set of column vectors - A set of equations (rows) to be solved - (your example: parity equations for coding theory) - The covariance of elements in a vector space - The Hessian of a function for numerical optimization - The adjacency representation of a graph - Just a 2D image (compression algorithms) ... (I'…

The first three can reasonably be thought of as defining linear transformations. For linear systems of equations A x = b, x is an unknown vector in the input space that is mapped by A to b. Both covariance matrices and Hessians are more naturally thought of as tensors, not matrices (and therefore not linear transformations). That is, they take in two vectors as input and produce a single real number as output. As for…

If you squint hard enough, you can see all of them as linear transformations (even the 2D images :-).

I politely disagree about covariance and Hessians. I can squint and say that the Hessian provides a change in gradient when multiplied by a delta vector. Similarly for covariance... Or you could look at it as one half of the dot product for a Bhattacharyya distance, which is just a product of three matrices (row vector, square matrix, col vector). No need for tensors yet.

That is unless you decide to squint hard enough to see everything as tensors! :-)

Re: An Intuitive Guide to Linear Algebra (2012)

#57

Post by jackthetab »

adamcharnock wrote: post #36

I... I still really struggle with this. I'm a smart person, I've got a bachelors of engineering, I've been a professional software developer for around 14 years now, and I've built a house. But there is something about degree-level maths and beyond that I find deeply unintuitive in a way that software development isn't. Through comments here I found 3blue1brown's (clearly much loved) videos. By the third video I was…

> But there is something about degree-level maths and beyond that I find deeply unintuitive in a way that software development isn't.

Grok.

Over the years, I come to the conclusion that one of the stumbling blocks is the definition/concept of "application". Just like the definition of "theory" is different for a layman ("My theory is..." == "My guess is...") than from a scientist's definition ("My theory is" == "My logical framework which incorporates all of the available data is..."), so the definition of "application" is different between mathematicians and engineers.

I've noticed that math books with titles like "$HIGHER_ORDER_MATH with Applications" means "$HIGHER_ORDER_MATH with Exercises". What I'm looking for is something like "$HIGHER_ORDER_MATH with Real-World Uses".

I've known LA for decades but, like you, where would I use it in my life? The turning point for me was Andrew Ng's Deep Neural Network course.

I knew that a DNN is a program of matrix operations, but how do you get 5,000 images into a matrix? One way is to resize all the images to the same n x n size, take the first pixel of each picture and break them into their RGB components. You now have the first three rows of your input matrix. Repeat for all other pixels and voila! You have a 5,000 x n matrix that you can do linear algebra on! _That's_ an application; having me add two matrices together is an exercise.

Since that insight, I've used LA in my job in the hospitably sector with impressive success because now I know how to apply it. Math books and 3B1B show you the math. We engineers (or at least this one) need real world uses.

Re: An Intuitive Guide to Linear Algebra (2012)

#58

Post by andrepd »

Goes to show how different people have different tastes. I find this type of exposition very confusing and very unenlightening. Give me a Landau-style "minimalistic"/"focused" explanation any day. Not to mention, it tries so hard to simplify things to a simple analogy (the spreadsheet thing) that it ends up being plain misleading. In other words: "Make things as simple as then can be, but no simpler."

Re: An Intuitive Guide to Linear Algebra (2012)

#59

Post by gfxgirl »

adamcharnock wrote: post #36

I... I still really struggle with this. I'm a smart person, I've got a bachelors of engineering, I've been a professional software developer for around 14 years now, and I've built a house. But there is something about degree-level maths and beyond that I find deeply unintuitive in a way that software development isn't. Through comments here I found 3blue1brown's (clearly much loved) videos. By the third video I was…

100% Agreed. I've written 3D engines in shipping games and yet I can't do math for shit. I've tried watching 3blue1brown and they have pretty pictures but they don't help at me all. I feel like they're mostly appreciated by people who already understand and can't remember what it's like to not understand.

I was recently watching videos and trying to read papers on geometric algebra and getting totally and utterly lost on actually applying it.

Re: An Intuitive Guide to Linear Algebra (2012)

#60

Post by MrK93 »

adamcharnock wrote: post #36

I... I still really struggle with this. I'm a smart person, I've got a bachelors of engineering, I've been a professional software developer for around 14 years now, and I've built a house. But there is something about degree-level maths and beyond that I find deeply unintuitive in a way that software development isn't. Through comments here I found 3blue1brown's (clearly much loved) videos. By the third video I was…

I'm currently working on a linear algebra heavy linear programming model for an optimization thesis and the general trick is, as with writing code, to improve iteration by iteration of the model. Starting with a complex mathematical model is always a bad idea, so start small and iteratively improve it. This means no use of advanced concepts unless needed. Also, nobody uses every bit of math in the everyday work. As with programming, you just need to understand the general concepts and the rest can be figured out step by step.
Post reply on HN