Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eigenvectors and Eigenvalues (2015)

setosa.io

Post reply on HN
Print view View original

41–50 of 87 posts

Re: Eigenvectors and Eigenvalues (2015)

#41

Post by Bukhmanizer »

xevb3k wrote: post #40
Bukhmanizer wrote: post #31

Earlier quoted context omitted.

It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just…

Interesting, well I’ll try to keep reviewing this stuff and hoping I find an application. I really would like to find an application in my work, because without that I find new techniques don’t really stick and after a few months I forget them...

I'm the exact same way, my job is really heavy in linear algebra, so it sticks more easily for me.

Usually I go through the code and ask "what am I trying to do here" and "can I do this a better way". A lot of the aforementioned speedups have come because the previous developer was obviously trying to do something, like create a linear projector, but were following some sort of math formula, so made a bunch of extraneous matrices that were huge.

It's a simple fix, but adds up when you're dealing with massive datasets.

Re: Eigenvectors and Eigenvalues (2015)

#43

Post by lewis500 »

Interesting to see this back on the front page after three years. Still remember us sitting in our living room drawing this on paper and arguing about the right approaches.

Maybe one day vicapow and I will make a triumphant return to the explorables space, but life has a way of getting in the way as you get older.

Re: Eigenvectors and Eigenvalues (2015)

#44

Post by dang »

lewis500 wrote: post #43

Interesting to see this back on the front page after three years. Still remember us sitting in our living room drawing this on paper and arguing about the right approaches. Maybe one day vicapow and I will make a triumphant return to the explorables space, but life has a way of getting in the way as you get older.

That was https://news.ycombinator.com/item?id=8918259.

Re: Eigenvectors and Eigenvalues (2015)

#46

Post by ItsMe000001 »

angersock wrote: post #39
WhompingWindows wrote: post #15

Earlier quoted context omitted.

Why should you know what they do and why they're important? How does that practically change my R code?

Because, say, knowing about Fourier transforms can help you write more efficient filtering or open up new ways to view your data--perhaps there's a really interesting behavior in the frequency domain you'd miss otherwise. If you just want to be a statistical script kiddie you do you. :)

When working with real world data almost everything is more important than being able to use the most abstract methods "to extract the last bit of data". It's often extremely fuzzy to begin with, the collection process to what it represents, for me, while I love math and see it as the "magical language" in a magical world, I find common sense and a certain kind of work ethics go soooo much further than any math Ph.D. I (with a CS degree) wanted to start another study (over a decade later, thirsting for new theory and new knowledge) and chose math - but that was around the same time edX and Coursera etc. got up and running with loads of courses, and I ended up ditching the additional math degree for loads and loads of courses in fields like medicine and biology, deciding to go for breadth instead. Of course, that's purely personal and even as an anecdote not worth much to anyone. It's just that I too got disillusioned with (higher and higher) math as an helpful tool in practical life. Of course I still see the benefits in many fields, but I think a surprisingly low number of very good specialists may be all we need. The rest of us can just ask them when we actually do need something.

Right now I'm taking a "math. modeling" course. Still, the only use case I ever found was... other courses! I already modeled a little bit in a biology course. Sure, in real life I could model this or that, but the truth is that a very rough estimate guided by experience and "feeling" has always been enough. There are too many variables that cannot be accurately measured, so going for a nice model is kind of useless.

For example, I was just asked today about the performance of the crypto-hash-connected data storage and exchange library I wrote. Now that sounds like something I could model! Only experience tells me that's useless. The only worthwhile answer is to set up a concrete scenario, with a concrete app using it, concrete network and concrete systems, and test it. Could be anything from smartphones to well-connected servers. Sure I could create a sophisticated model and simulation - and it would be useless.

Maybe I'm just a bit, or more than just a bit, disappointed that all the considerable amount of math I learned in my life didn't seem to be of nearly as much use as I would have hoped. I'm also frustrated each time such a topic comes up and everyone is so excited about how great it is, and I always feel like I'm missing something despite trying hard, like the color blind guy looking at paintings. I mean the usefulness to me, not understanding it.

Re: Eigenvectors and Eigenvalues (2015)

#47

Post by zawerf »

soVeryTired wrote: post #21
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variab…

I use the 2nd point a lot for debugging 3d transforms. To expand upon it, for example in three dimensions the three axes are:

(1, 0, 0)

(0, 1, 0)

(0, 0, 1)

To find out where those axes are after a 3x3 matrix transform, you just read off the first, second, and third columns of the matrix respectively. Then you can mentally visualize another unit cube in the new coordinate system using those three vectors as the edges of the cube.

Really basic change-of-basis stuff but academic lectures don't emphasize how useful it is to be able to look at a matrix of numbers immediately know what it does.

Re: Eigenvectors and Eigenvalues (2015)

#48

Post by qmalzp »

soVeryTired wrote: post #21
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variab…

I think the first point is only true for symmetric matrices (which includes those that show up in multivariable calc). In general, the eigenvectors need not be orthogonal.

Re: Eigenvectors and Eigenvalues (2015)

#49

Post by pphysch »

ItsMe000001 wrote: post #46
angersock wrote: post #39

Earlier quoted context omitted.

Because, say, knowing about Fourier transforms can help you write more efficient filtering or open up new ways to view your data--perhaps there's a really interesting behavior in the frequency domain you'd miss otherwise. If you just want to be a statistical script kiddie you do you. :)

When working with real world data almost everything is more important than being able to use the most abstract methods "to extract the last bit of data". It's often extremely fuzzy to begin with, the collection process to what it represents, for me, while I love math and see it as the "magical language" in a magical world, I find common sense and a certain kind of work ethics go soooo much further than any math Ph.D.…

As a younger person (finishing up a Math BS) this resonates with my perspective.

IMHO, it comes down to individual beliefs about mathematical realism. Is there anything inherently real about math, or is it just a man-made, arbitrary set of cognitive tools? Is it valid to presume the existence of a Grand Mathematical Framework that can solve any problem a priori? Or, is every problem unique and independent of mathematical developments?

From the little I've read about Math history, it seems pretty clear that the Problems came first, and the Mathematics followed. Infintesimal calculus, game theory, etc. were mathematical ideas developed primarily to solve real problems. Then 20th century formalism came along and rebranded much of mathematics under a "clean" framework, while giving little attention to the human environment in which much of it was developed.

To me, it is a great shame that abstract mathematical concepts are made further abstract (e.g. in math education) by distancing them from their human roots. Instead of forcing oneself to understand this mathematical "new testament", I think it's far more productive to adopt this sort of irreverent attitude towards math as you describe.

Einstein:

>"As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality."

Re: Eigenvectors and Eigenvalues (2015)

#50

Post by aje403 »

pphysch wrote: post #49
ItsMe000001 wrote: post #46

Earlier quoted context omitted.

When working with real world data almost everything is more important than being able to use the most abstract methods "to extract the last bit of data". It's often extremely fuzzy to begin with, the collection process to what it represents, for me, while I love math and see it as the "magical language" in a magical world, I find common sense and a certain kind of work ethics go soooo much further than any math Ph.D.…

As a younger person (finishing up a Math BS) this resonates with my perspective. IMHO, it comes down to individual beliefs about mathematical realism. Is there anything inherently real about math, or is it just a man-made, arbitrary set of cognitive tools? Is it valid to presume the existence of a Grand Mathematical Framework that can solve any problem a priori? Or, is every problem unique and independent of mathemat…

Perhaps colleges can start adding an "applied" math major with a focus on subjects that more directly involve the human environment more directly to alleviate the problem you're describing?
Post reply on HN