Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eigenvectors and Eigenvalues (2015)

setosa.io

Post reply on HN
Print view View original

31–40 of 87 posts

Re: Eigenvectors and Eigenvalues (2015)

#31

Post by Bukhmanizer »

xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just by knowing how to do the same thing in a more efficient way.

My feeling is having a basic knowledge of testing/caching/memory management is way more useful when you're doing large image analysis.

Re: Eigenvectors and Eigenvalues (2015)

#32

Post by xevb3k »

aje403 wrote: post #11
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

This is frightening but believable. I've worked with a few "quants" who stared at me doe eyed explaining eigen* and basic calculus concepts to them in the context of why their calculations don't add up. You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.

In the context I used it, FFT wasn’t really of fundamental importance. Using FFT (and in particular FFTW) gave a performance improvement (execution speed), but no real advantage in terms of accuracy over an alternative naive method...

So... yes I guess I’ve just not seen anywhere in my work where this stuff has proved useful...

Re: Eigenvectors and Eigenvalues (2015)

#34

Post by sakuronto »

soVeryTired wrote: post #21
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variab…

Not a very useful addition but hypercube is to cube as parallelotope is to parallelepiped.

Re: Eigenvectors and Eigenvalues (2015)

#35

Post by whatshisface »

electricslpnsld wrote: post #25
whatshisface wrote: post #19

Earlier quoted context omitted.

As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have…

> So a rotation would have no eigenvectors Rotations have eigenvectors: a 2D rotation has two complex eigenvectors, a 3D rotation has one real and two complex eigenvectors, ...

That's a fair and true catch, but I can cover myself by pointing out that the article was only talking about matrices in R^(m x n). ;)

Re: Eigenvectors and Eigenvalues (2015)

#36

Post by aje403 »

xevb3k wrote: post #32
aje403 wrote: post #11

Earlier quoted context omitted.

This is frightening but believable. I've worked with a few "quants" who stared at me doe eyed explaining eigen* and basic calculus concepts to them in the context of why their calculations don't add up. You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.

In the context I used it, FFT wasn’t really of fundamental importance. Using FFT (and in particular FFTW) gave a performance improvement (execution speed), but no real advantage in terms of accuracy over an alternative naive method... So... yes I guess I’ve just not seen anywhere in my work where this stuff has proved useful...

It's pretty much ubiquitous in any quantitative field... would not even know where to start

Re: Eigenvectors and Eigenvalues (2015)

#37

Post by sakuronto »

electricslpnsld wrote: post #25
whatshisface wrote: post #19

Earlier quoted context omitted.

As someone who understands eigenfunctions already, I don't understand the pictures either. Here is the best way to think about it: a matrix is a transformation, a composition of rotation, scaling, etc. Eigensets are lines going through the origin that the matrix moves points along. So a rotation would have no eigenvectors because none of the points move in a straight line, while a scaling along the x axis would have…

> So a rotation would have no eigenvectors Rotations have eigenvectors: a 2D rotation has two complex eigenvectors, a 3D rotation has one real and two complex eigenvectors, ...

When your field of interest is the reals, those complex eigenvectors don't matter.

Re: Eigenvectors and Eigenvalues (2015)

#38

Post by AceJohnny2 »

steamer25 wrote: post #5

3Blue1Brown has a good series on YouTube for building intuition in linear algebra: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x... In one of the last videos in the (relatively short) series, he discusses eigen-*: ~'eigen-stuffs are straight-forward but only make sense if you have a solid visual understanding of the pre-requisites (linear transformations, determinants, linear systems of equations, chan…

3Blue1Brown (Grant Sanderson) is really, really good. I follow a number of education channels on YouTube, and Grant blows them all out of the water for the kind of insights, new perspectives, and inspiration he provides. His animations are fantastically put together to clearly and unobtrusively illustrate the point he's making. I also really like his voice, soothing, clear and with enough intonation to avoid boredom, and perfect pace. I wish I had his linear algebra series back in college, I suspect I would have done much better.

He's the creator I support the most on Patreon: https://www.patreon.com/3blue1brown

Re: Eigenvectors and Eigenvalues (2015)

#39

Post by angersock »

WhompingWindows wrote: post #15
jdonaldson wrote: post #7

Earlier quoted context omitted.

Eigenvectors and Eigenvalues show up everywhere, although sometimes it's in the form of an iterative estimate (PageRank is basically the power method estimation of the first eigenvector of a connected graph of web pages). They're in the same class as logarithms and Fourier transforms IMHO. You won't need to calculate them by hand, but you should know what they do and why they're important.

Why should you know what they do and why they're important? How does that practically change my R code?

Because, say, knowing about Fourier transforms can help you write more efficient filtering or open up new ways to view your data--perhaps there's a really interesting behavior in the frequency domain you'd miss otherwise.

If you just want to be a statistical script kiddie you do you. :)

Re: Eigenvectors and Eigenvalues (2015)

#40

Post by xevb3k »

Bukhmanizer wrote: post #31
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just…

Interesting, well I’ll try to keep reviewing this stuff and hoping I find an application.

I really would like to find an application in my work, because without that I find new techniques don’t really stick and after a few months I forget them...

Post reply on HN