Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eigenvectors and Eigenvalues (2015)

setosa.io

Post reply on HN
Print view View original

21–30 of 87 posts

Re: Eigenvectors and Eigenvalues (2015)

#21

Post by soVeryTired »

xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

For a large fraction of probability theory, you only need two main facts from linear algebra.

First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors.

Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform.

That more or less covers covariances, PCA, and change of variables. Whenever I try to understand or re-derive a fact in probability, I almost always end up back at one or the other fact.

They're also useful in multivariate calculus, which is really just stitched-together linear algebra.

Re: Eigenvectors and Eigenvalues (2015)

#23

Post by madengr »

jdonaldson wrote: post #7
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

Eigenvectors and Eigenvalues show up everywhere, although sometimes it's in the form of an iterative estimate (PageRank is basically the power method estimation of the first eigenvector of a connected graph of web pages). They're in the same class as logarithms and Fourier transforms IMHO. You won't need to calculate them by hand, but you should know what they do and why they're important.

I use logarithms and FFTs daily, but got a C in Linear Algebra, after getting A in all other EE/Math/Phys courses. It kicked my ass once we got to proofs.

Re: Eigenvectors and Eigenvalues (2015)

#24

Post by pizza »

haskellandchill wrote: post #16

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to gras…

Does this gif help? https://commons.wikimedia.org/wiki/File:Eigenvectors.gif

Re: Eigenvectors and Eigenvalues (2015)

#25

Post by electricslpnsld »

whatshisface wrote: post #19
haskellandchill wrote: post #16

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to gras…

As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have…

> So a rotation would have no eigenvectors

Rotations have eigenvectors: a 2D rotation has two complex eigenvectors, a 3D rotation has one real and two complex eigenvectors, ...

Re: Eigenvectors and Eigenvalues (2015)

#26

Post by msla »

whatshisface wrote: post #19
haskellandchill wrote: post #16

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to gras…

As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have…

> So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have an eigenset that was also along the x axis, consisting of the points that were moved straight up or down.

Maybe I'm being pedantic, but rotation matrices have eigenvectors and eigenvalues, but the eigenvalues are imaginary because imaginary numbers are rotations in the complex plane.

It's exactly like saying x^2 = -1 has no solutions: It has two solutions, like any other quadratic, but neither of them are real.

In three dimensions, rotation matrices have three eigenvalues, one of them being 1, and the eigenvector corresponding to that eigenvalue is, naturally, the rotation axis.

https://en.wikipedia.org/wiki/Rotation_matrix

Re: Eigenvectors and Eigenvalues (2015)

#28

Post by haskellandchill »

pizza wrote: post #24
haskellandchill wrote: post #16

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to gras…

Does this gif help? https://commons.wikimedia.org/wiki/File:Eigenvectors.gif

No gif will help. There is no visual explanation that will help was my point.

Re: Eigenvectors and Eigenvalues (2015)

#29

Post by haskellandchill »

whatshisface wrote: post #19
haskellandchill wrote: post #16

The visual explanation movement falls flat for me. It's like trying to understand Monads through blog posts. It's great if you already understand the concept to develop your intuition, or if you've never heard of the concept to pique your interest, but it won't help in the intermediate area where you know what you want to know but don't understand it fully. I need to build proofs through incremental exercises to gras…

As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have…

I want to know it. I have to manipulate the mathematical objects computationally using proof techniques to know it. That just takes time. Thanks though.

Re: Eigenvectors and Eigenvalues (2015)

#30

Post by sannee »

Eigen{vectors,values} seemed like this totally arbitrary concept when I first learned about them. Later it turned out that they are actually really awesome and pop up all the time.

Multivariable function extrema? Just look at the eigenvalues of the hessian. Jacobi method convergence? Eigenvalues of the update matrix. RNN gradient explosion? Of course, eigenvalues.

Post reply on HN