Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Matrices can be your friends (2002)

sjbaker.org

Post reply on HN
Print view View original

81–90 of 115 posts

Re: Matrices can be your friends (2002)

#81

Post by nyrikki »

Ono-Sendai wrote: post #45
nyrikki wrote: post #38

Earlier quoted context omitted.

> You can do rotation with a 3x3 matrix. You can do a rotation or some rotations but SO(3) is not simply connected. It mostly works for rigid bodies centered on the origin, but gimbal lock or Dirac's Plate Trick are good counter example lenses. Twirling a baton or a lasso will show that 720 degrees is the invariant rotation in SO(3) The point at infinity with a 4x4 matrix is one solution, SU(3), quaternions, or recen…

I think you are confused about what 'simply connected' means. A 3x3 matrix can represent any rotation. Also from a given rotation there is a path through the space of rotations to any other rotation. It's just that some paths can't be smoothly mapped to some other paths.

SO(3) contains all of the orthogonal 3x3 matrices of determinant 1.

If you are dealing with rigid bodies rotated though the origin like with the product of linear translations you can avoid the problem. At least with an orthonormal basis R^3 with an orthogonal real valued 3x3 matrix real entries which, where the product of it with its transpose produces the identity matrix and with determinant 1

But as soon as you are dealing with balls, where the magnitude can be from the origin to the radius, you run into the issue that the antipodes are actually the same point, consider the north and south poles being the same point, that is what I am saying when the topology is not simply connected.

The rigid body rotation about the origin is just a special case.

Twist a belt twice and tape one end to the table and you can untwist it with just horizontal translation, twist it once (360deg) and you cannot.

Re: Matrices can be your friends (2002)

#82

Post by notmyjob »

1980phipsi wrote: post #50
notmyjob wrote: post #35

Earlier quoted context omitted.

Math achievement correlates strongly with visuospatial reasoning. Programmers may not be as proficient in math as economists, but they are better at it than biologists or lawyers.

I would distinguish between visual imagination and visuospatial reasoning. For people like myself with aphantasia, there are often problems solving strategies that can help you when you can’t visualize. Like draw a picture. And lots of problems don’t really require as much visual imagination as you would think. I’m pretty good at math, programming, and economics. Not top tier, but pretty good. If there are problems o…

It sounds like you have routed around your spatial visualization deficit, but that just proves the importance of alternate cognitive strategies rather than indicate that such an aptitude or deficit doesn’t ceteris paribus impact mathematical achievement.

https://en.wikipedia.org/wiki/Spatial_visualization_ability

You probably are high g (iq), which has, historically at least, dominated other factors in determining overall outcomes.

Re: Matrices can be your friends (2002)

#83

Post by notmyjob »

neffy wrote: post #44
notmyjob wrote: post #35

Earlier quoted context omitted.

Math achievement correlates strongly with visuospatial reasoning. Programmers may not be as proficient in math as economists, but they are better at it than biologists or lawyers.

And since the economist's main skill at math is fitting a very short ruler to a very large curve... i wouldn't put them ahead of lawyers...

There are economists and there are economists. I doubt Pam Bondi was top in real analysis or other college level maths. Maybe, but I doubt it.

Re: Matrices can be your friends (2002)

#84

Post by recursive »

omnicognate wrote: post #78
recursive wrote: post #66

Earlier quoted context omitted.

I took a linear algebra class, as well as many others. It didn't work. Most math classes I've taken granted me some kind of intuition for the subject material. Like I could understand the concept independent from the name of the thing. In linear algebra, it was all a series of arbitrary facts without reason for existing. I memorized them for the final exam, and probably forgot them all the next day, as they weren't a…

You might want to try Linear Algebra Done Right by Sheldon Axler. It's a short book, succinct but extremely clear and approachable. It explains Linear Algebra without using determinants, which are relegated to the end, and emphasises understanding the powerful ideas underpinning the subject rather than learning seemingly arbitrary manipulations of lists and tables of numbers. Those manipulations are of course extreme…

The page count suggests that we have different ideas of what's meant by "short". In any case, it looks great from the forewords. If I ever want to make a serious try to really get it, this is probably what I'll use.

Re: Matrices can be your friends (2002)

#85

Post by inamberclad »

shiandow wrote: post #3

I don't think there's any mathematical reason to lay out the elements in memory that way. Sure given no context I would probably use i = row + n col as index, but it doesn't really matter much me. If I had to pick between a matrix being a row of vectors or a column of covectors, I'd pick the latter. And M[i][j] should be the element in row i column j, which is nonnegotiable.

Matlab deliberately notes that its matrices are laid out like this since most matrix operations occur on columns, a whole column can be loaded on a cache line.

Re: Matrices can be your friends (2002)

#86

Post by getnormality »

recursive wrote: post #84
omnicognate wrote: post #78

Earlier quoted context omitted.

You might want to try Linear Algebra Done Right by Sheldon Axler. It's a short book, succinct but extremely clear and approachable. It explains Linear Algebra without using determinants, which are relegated to the end, and emphasises understanding the powerful ideas underpinning the subject rather than learning seemingly arbitrary manipulations of lists and tables of numbers. Those manipulations are of course extreme…

The page count suggests that we have different ideas of what's meant by "short". In any case, it looks great from the forewords. If I ever want to make a serious try to really get it, this is probably what I'll use.

It is widely considered to deliver on the promise of done right!

Re: Matrices can be your friends (2002)

#88

Post by owlbite »

bee_rider wrote: post #54
owlbite wrote: post #43

Earlier quoted context omitted.

What I suspect he really means is that FORTRAN lays out its arrays column-major, whilst C choose row-major. Historically most math software was written in the former, including the de facto standard BLAS and LAPACK APIs used for most linear algebra. Mix-and-matching memory layouts is a recipe for confusion and bugs, so "mathematicians" (which I'll read as people writing a lot of non-ML matrix-related code) tend to pr…

The MKL blas/lapack implementation also provides the “cblas” interface (I’m sure most blas implementations do, I’m just familiar with MKL—BLIS seems quite willing to provide additional interfaces to I bet they provide it as well) which explicitly accepts arguments for row or column ordering. Internally the matrix is tiled out anyway (for gemm at least) so column vs row ordering is probably a little less important now…

Oh yes, from an actual implementation POV you can just apply some transpose and ordering transforms to convert from row major to column major or vice-versa. cblas is pretty universal though I don't think any LAPACK C API ever gained as wide support for non column-major usage (and actually has some routines where you can't just pull transpose tricks for the transformation).

Certain layouts have performance advantages for certain operations on certain microarchitectures due to data access patterns (especially for level 2 BLAS), but that's largely irrelevant to historical discussion of the API's evolution.

Re: Matrices can be your friends (2002)

#89

Post by dekhn »

notmyjob wrote: post #82
1980phipsi wrote: post #50

Earlier quoted context omitted.

I would distinguish between visual imagination and visuospatial reasoning. For people like myself with aphantasia, there are often problems solving strategies that can help you when you can’t visualize. Like draw a picture. And lots of problems don’t really require as much visual imagination as you would think. I’m pretty good at math, programming, and economics. Not top tier, but pretty good. If there are problems o…

It sounds like you have routed around your spatial visualization deficit, but that just proves the importance of alternate cognitive strategies rather than indicate that such an aptitude or deficit doesn’t ceteris paribus impact mathematical achievement. https://en.wikipedia.org/wiki/Spatial_visualization_ability You probably are high g (iq), which has, historically at least, dominated other factors in determining ov…

I took some sort of IQ test when I was a kid and there was an entire section that was "if you rotate this object around that axis, it matches which of the followin g options". Try as I might, I can't picture this in my head (picturing anything other than a sphere or a cube is tough) but I found that I could look at the options and logically exclude them in a very tedious way by inspection.

It's one of the reasons I like computer graphics so much: the computer does the rotation for you! Stereo graphics (using the funny LCD glasses) was a true revelation to me, and learning how to rotate things using matrics was another.

Re: Matrices can be your friends (2002)

#90

Post by Const-me »

owlbite wrote: post #43
Viliam1234 wrote: post #5

> Mathematicians like to see their matrices laid out on paper this way (with the array indices increasing down the columns instead of across the rows as a programmer would usually write them). Could a mathematician please confirm of disconfirm this? I think that different branches of mathematics have different rules about this, which is why careful writers make it explicit.

What I suspect he really means is that FORTRAN lays out its arrays column-major, whilst C choose row-major. Historically most math software was written in the former, including the de facto standard BLAS and LAPACK APIs used for most linear algebra. Mix-and-matching memory layouts is a recipe for confusion and bugs, so "mathematicians" (which I'll read as people writing a lot of non-ML matrix-related code) tend to pr…

> is written in languages that inherited their array ordering from C

It’s not just C. Modern GPU hardware only supports row major memory layout for 2D and 3D textures (ignoring specialized layouts like swizzling and block compression but none of them are column major either). Modern image and video codecs only support row major layout for bitmaps.

Post reply on HN