Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Eigenvectors and Eigenvalues (2015)

setosa.io

Post reply on HN
Print view View original

51–60 of 87 posts

Re: Eigenvectors and Eigenvalues (2015)

#51

Post by gh02t »

xevb3k wrote: post #40
Bukhmanizer wrote: post #31

Earlier quoted context omitted.

It's one of those things that you don't notice when it's missing, but probably would help a bit if you knew it. That being said, I have to deal with linear algebra every day, and aside from proofs (which obviously they help with), there have been maybe a handful of times that having a deep knowledge of eigenvectors and eigenvalues has helped significantly. Once or twice though, I've got massive speedups (>500x) just…

Interesting, well I’ll try to keep reviewing this stuff and hoping I find an application. I really would like to find an application in my work, because without that I find new techniques don’t really stick and after a few months I forget them...

It depends on the field you're in. For example, if you're in an area that heavily uses differential equations (many engineering disciplines) then you're probably gonna be using eigenvectors a lot, as they are important for solving a lot of problems. Other areas may not need them at all. It also depends on your depth in the field. A rank and file engineer may not need to know anything about them - they underpin a lot of numerical methods, but get hidden away in software packages. Someone developing those software packages likely will, though. Techniques based on eigenvectors and eigenvalues are extremely important in my field (nuclear engineering... you've probably heard the term "critical", that refers to an eigenvalue), but I know someone who is an excellent civil engineer and knows next to nothing about them (or linear algebra in general) because they aren't that important for what he works on.

Forgetting stuff you don't use is pretty normal, the important thing is to be able to recognize when a technique you don't remember the details of might be applicable, and to know where to look to refresh your memory.

Re: Eigenvectors and Eigenvalues (2015)

#52

Post by bashcorrupt »

AceJohnny2 wrote: post #38
steamer25 wrote: post #5

3Blue1Brown has a good series on YouTube for building intuition in linear algebra: https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2x... In one of the last videos in the (relatively short) series, he discusses eigen-*: ~'eigen-stuffs are straight-forward but only make sense if you have a solid visual understanding of the pre-requisites (linear transformations, determinants, linear systems of equations, chan…

3Blue1Brown (Grant Sanderson) is really, really good. I follow a number of education channels on YouTube, and Grant blows them all out of the water for the kind of insights, new perspectives, and inspiration he provides. His animations are fantastically put together to clearly and unobtrusively illustrate the point he's making. I also really like his voice, soothing, clear and with enough intonation to avoid boredom,…

>I follow a number of education channels on YouTube

Any recommendations? I would love to look into channels that you think are like 3Blue1Brown but in other subjects (natural sciences, history, art etc.).

Re: Eigenvectors and Eigenvalues (2015)

#53

Post by soVeryTired »

qmalzp wrote: post #48
soVeryTired wrote: post #21

Earlier quoted context omitted.

For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variab…

I think the first point is only true for symmetric matrices (which includes those that show up in multivariable calc). In general, the eigenvectors need not be orthogonal.

Yep, you could well be right. The image of an ellipse under a linear transform is definitely an ellipse, but I'm not sure about the eigenvectors in the general case.

The symmetric case is by far the most relevant for probability theory though.

Re: Eigenvectors and Eigenvalues (2015)

#54

Post by akvadrako »

aje403 wrote: post #50
pphysch wrote: post #49

Earlier quoted context omitted.

As a younger person (finishing up a Math BS) this resonates with my perspective. IMHO, it comes down to individual beliefs about mathematical realism. Is there anything inherently real about math, or is it just a man-made, arbitrary set of cognitive tools? Is it valid to presume the existence of a Grand Mathematical Framework that can solve any problem a priori? Or, is every problem unique and independent of mathemat…

Perhaps colleges can start adding an "applied" math major with a focus on subjects that more directly involve the human environment more directly to alleviate the problem you're describing?

Applied math is definitely a major. Do you mean something else?

Re: Eigenvectors and Eigenvalues (2015)

#56

Post by aidos »

zawerf wrote: post #47
soVeryTired wrote: post #21

Earlier quoted context omitted.

For a large fraction of probability theory, you only need two main facts from linear algebra. First, linear transforms map spheres to ellipsoids. The axes of the ellipsoid are the eigenvectors. Second, linear transforms map (hyper) cubes to parallelpipeds. If you start with a unit cube, the volume of the parallelpiped is the determinant of the transform. That more or less covers covariances, PCA, and change of variab…

I use the 2nd point a lot for debugging 3d transforms. To expand upon it, for example in three dimensions the three axes are: (1, 0, 0) (0, 1, 0) (0, 0, 1) To find out where those axes are after a 3x3 matrix transform, you just read off the first, second, and third columns of the matrix respectively. Then you can mentally visualize another unit cube in the new coordinate system using those three vectors as the edges…

This concept totally changed my intuitive understanding of matrices. Beautifully illustrated in the below 3blue1brown video.

https://youtu.be/kYB8IZa5AuE?t=3m15s

Re: Eigenvectors and Eigenvalues (2015)

#57

Post by AceJohnny2 »

bashcorrupt wrote: post #52
AceJohnny2 wrote: post #38

Earlier quoted context omitted.

3Blue1Brown (Grant Sanderson) is really, really good. I follow a number of education channels on YouTube, and Grant blows them all out of the water for the kind of insights, new perspectives, and inspiration he provides. His animations are fantastically put together to clearly and unobtrusively illustrate the point he's making. I also really like his voice, soothing, clear and with enough intonation to avoid boredom,…

>I follow a number of education channels on YouTube Any recommendations? I would love to look into channels that you think are like 3Blue1Brown but in other subjects (natural sciences, history, art etc.).

Edit: I realize I overlooked the "like 3Blue1Brown" prereq, instead sharing a list of the educational/interesting channels I find worthwhile. The most like 3Blue1Brown will be the PBS ones (especially SpaceTime), MinutePhysics, and Mathologer, for using diagrams to convey abstract concepts.

PBS Space Time and Eons are both awesome:

* PBS Space Time, covers cosmology and quantum physics:

https://www.youtube.com/channel/UC7_gcs09iThXybpVgjHZ_7g

* PBS Eons, for geology and paleontology:

https://www.youtube.com/channel/UCzR-rom72PHN9Zg7RML9EbA

* Smarter Every Day, Destin's enthusiasm is contagious:

https://www.youtube.com/user/destinws2

* Extra Credits various topics (video game design, History, and recently history of Sci-Fi) are great:

https://www.youtube.com/user/ExtraCreditz

* Today I Found Out is just on this side of clickbaity, and is this age's "Ripley's Believe It Or Not", but still interesting and more importantly well researched:

https://www.youtube.com/user/TodayIFoundOut

* Crash Course, of course:

https://www.youtube.com/channel/UCX6b17PVsYBQ0ip5gyeme-Q

* Gaming Historian, for some insight into the making of systems that formed my (and earlier) childhood:

https://www.youtube.com/channel/UCnbvPS_rXp4PC21PG2k1UVg

* Minute Physics, whose latest few videos made Special Relativity understandable to this peon:

https://www.youtube.com/user/minutephysics/videos

* Practical Engineering, for some insight into civil engineering topics that we take for granted:

https://www.youtube.com/channel/UCMOqf8ab-42UUQIdVoKwjlQ

* Real Engineering, for insight into various other mechanical engineering topics:

https://www.youtube.com/channel/UCR1IuLEqb6UEA_zQ81kwXfg

* Standup Maths, host Matt Parker was the first to make maths approachable for me again (before 3Blue1Brown took the lead):

https://www.youtube.com/channel/UCSju5G2aFaWMqn-_0YBtq5A

* Steve Mould, who covers various topics both mathematical and physical. You may have seen that gif of him demonstrating the "levitating" siphoning "pearl necklace" (also a friend of Matt Parker, above):

https://www.youtube.com/channel/UCEIwxahdLz7bap-VDs9h35A

* The 8-Bit Guy, for some history of early home computer systems:

https://www.youtube.com/channel/UC8uT9cgJorJPWu7ITLGo9Ww

* Numberphile, the second-greatest math channel (after 3Blue1Brown), whose recent video finally made me take the Golden Ratio seriously, rather than an architectural gimmick/conspiracy theory:

https://www.youtube.com/user/numberphile

* Mathologer, another good math channel (but I must sheepishly admit I prefer 3Blue1Brown... sensing a pattern here?):

https://www.youtube.com/channel/UC1_uAIS3r8Vu6JjXWvastJg

* Periodic Videos, for chemistry and physics, often featuring the iconic Dr Martyn Poliakoff:

https://www.youtube.com/channel/UCtESv1e7ntJaLJYKIO1FoYw

* NileRed, for some homegrown chemistry, I particularly appreciate the candor of the approach and results:

https://www.youtube.com/user/TheRedNile

Not quite as much "educational", but still very very good:

* Every Frame A Painting, now finished, but a great explanation of what makes good cinematography:

https://www.youtube.com/channel/UCjFqcJQXGZ6T6sxyFB-5i6A

* NoClip, long-form documentaries about the making-of video games. Danny O'Dwyer is a treasure:

https://www.youtube.com/channel/UC0fDG3byEcMtbOqPMymDNbw

Apologies for the link spam, this list turned out longer than I expected as I went down my subscriptions, and I've probably missed a few worthy ones!

Edit the final: I discovered many of these channels through referrals from others I was watching, including from the twitter feeds of the authors. Turns out the educational landscape on YouTube is a well-connected graph!

Re: Eigenvectors and Eigenvalues (2015)

#58

Post by wglb »

xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

It is certainly frequent in engineering. If you need to analyze the stability of an electrical grid, there isn't much alternative.

A fun book on this is https://openlibrary.org/books/OL2398351M/The_algebraic_eigen...

Re: Eigenvectors and Eigenvalues (2015)

#59

Post by ajross »

aje403 wrote: post #11
xevb3k wrote: post #3

Whenever this kind of stuff comes up I feel like a bit of a fraud... I’ve written a bunch of scientific data analysis code. I have a science PhD. Written large image analysis pipelines that worked as well as the state of the art... been published etc. For the most part I’ve found basic math and heuristics to be good enough. Every so often I go relearn calculus. But honestly, none of this stuff ever seems to come in h…

This is frightening but believable. I've worked with a few "quants" who stared at me doe eyed explaining eigen* and basic calculus concepts to them in the context of why their calculations don't add up. You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.

> You mention you've used fourier transforms before - if you don't understand an eigenbasis then you don't have a fundamental understanding the math you're deploying.

That's a bit uncharitable. A fourier decomposition can absolutely be understood as an explicit bag of calculus tricks, with no loss of precision or generality. And an awful lot can be done with just those tools -- you don't need to explain JPEG compression or VLBI astronomy in terms of eigenvectors, for example.

Obviously (heh, "obviously") it's true that the space of decomposed functions form an orthogonal basis, so technically we're "really" operating in a linear space and that has expressive power too. But there are lots of ways of looking at problems.

To wit, you're not wrong. You're just... Well, you know.

Re: Eigenvectors and Eigenvalues (2015)

#60

Post by Jack000 »

ItsMe000001 wrote: post #46
angersock wrote: post #39

Earlier quoted context omitted.

Because, say, knowing about Fourier transforms can help you write more efficient filtering or open up new ways to view your data--perhaps there's a really interesting behavior in the frequency domain you'd miss otherwise. If you just want to be a statistical script kiddie you do you. :)

When working with real world data almost everything is more important than being able to use the most abstract methods "to extract the last bit of data". It's often extremely fuzzy to begin with, the collection process to what it represents, for me, while I love math and see it as the "magical language" in a magical world, I find common sense and a certain kind of work ethics go soooo much further than any math Ph.D.…

well you don't need this stuff until you do - some things still need definite, analytical performance guarantees.

I'd be pretty nervous riding an airplane that didn't use modern control theory, or going over a bridge that didn't use FEA - or an self-driving car that ran on a raspberry pi instead of a RTOS...

Post reply on HN