Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Matrices can be your friends (2002)

sjbaker.org

Post reply on HN
Print view View original

51–60 of 115 posts

Re: Matrices can be your friends (2002)

#51

Post by omnicognate »

treyd wrote: post #31
foofoo12 wrote: post #19

Earlier quoted context omitted.

> most programmers are visual thinkers I remember reading that there's a link between aphantasia (inability to visualize) and being on the spectrum. Being an armchair psychologist expert with decades of experience, I can say with absolute certainty that a lot of programmers are NOT visual thinkers.

This is interesting because, to me, programing is a deeply visual activity. It feels like wandering around in a world of forms until I find the structures I need and actually writing out the code is mostly a formality.

I would describe my experience of it similarly, but wouldn't call it "visual thinking" in the sense meant in the article, where one uses actual imagery and visual-spatial reasoning. Indeed, I almost completely lack the ability to conjure mental imagery (aphantasia) and I've speculated it might be because a part of my visual cortex is given over to the pseudo-visual activity that seems to take place when I program.

I'm especially sure my sort of pseudo-visual thinking isn't what the article means by "visual thinking" because I also use it when working through "piles of abstract math", which I take to very kindly indeed.

Is your "wandering" of this sort of pseudo-visual nature, or do you see actual visual images that could be drawn? Very intriguing if the latter, and I'd be curious to know what they look like.

Re: Matrices can be your friends (2002)

#52

Post by kxyvr »

Viliam1234 wrote: post #5

> Mathematicians like to see their matrices laid out on paper this way (with the array indices increasing down the columns instead of across the rows as a programmer would usually write them). Could a mathematician please confirm of disconfirm this? I think that different branches of mathematics have different rules about this, which is why careful writers make it explicit.

I'm an applied mathematician and this is the most common layout for dense matrices due to BLAS and LAPACK. Note, many of these routines have a flag to denote when working with a transpose, which can be used to cheat a different memory layout in a pinch. There are also parameters for increments in memory, which can help when computing across a row as opposed to down a column, which can also be co-opted. Unless there's a reason not to, I personally default to column major ordering for all matrices and tensors and use explicit indexing functions, which tends to avoid headaches since my codes are consistent with most others.

Abstractly, there's no such thing as memory layout, so it doesn't matter for things like proofs, normally.

Re: Matrices can be your friends (2002)

#53

Post by jesuslop »

kevindamm wrote: post #22
dgacmu wrote: post #17

Earlier quoted context omitted.

You can do rotation with a 3x3 matrix. The first lecture was using a 4x4 matrix because you can use it for a more general set of transformations, including affine transforms (think: translating an object by moving it in a particular direction). Since you can combine a series of matrix multiplications by just pre-multiplying the matrix, this sets you up for doing a very efficient "move, scale, rotate" of an object usi…

The first time I learned it was from a book by LaMothe in the 90s and it starts with your demonstration of 3D matrix transforms, then goes "ha! gimbal lock" then shows 4D transforms and the extension to projection transforms, and from there you just have an abstraction of your world coordinate transform and your camera transform(s) and most everything else becomes vectors. I think it's probably the best way to teach…

Think OpenGL used all those 2,3,4D critters at API level. It must be very hardware friendly to reduce your pipeline to matrix product. Also your scene graph (tree) is just this, you attach relative rotations and translations to graph nodes. You push your mesh (stream of triangles) at tree nodes, and composition of relative transforms up to the root is matrix product (or was the inverse?) that transform the meshes that go to the pipeline. For instance character skeletons are scene subgraphs, bones have translations, articulations have rotations. That's why it is so convenient to have rotations and translations in a common representation, and a linear one (4D matrix) is super. All this excluding materials, textures, and so on, I mean.

Re: Matrices can be your friends (2002)

#54

Post by bee_rider »

owlbite wrote: post #43
Viliam1234 wrote: post #5

> Mathematicians like to see their matrices laid out on paper this way (with the array indices increasing down the columns instead of across the rows as a programmer would usually write them). Could a mathematician please confirm of disconfirm this? I think that different branches of mathematics have different rules about this, which is why careful writers make it explicit.

What I suspect he really means is that FORTRAN lays out its arrays column-major, whilst C choose row-major. Historically most math software was written in the former, including the de facto standard BLAS and LAPACK APIs used for most linear algebra. Mix-and-matching memory layouts is a recipe for confusion and bugs, so "mathematicians" (which I'll read as people writing a lot of non-ML matrix-related code) tend to pr…

The MKL blas/lapack implementation also provides the “cblas” interface (I’m sure most blas implementations do, I’m just familiar with MKL—BLIS seems quite willing to provide additional interfaces to I bet they provide it as well) which explicitly accepts arguments for row or column ordering.

Internally the matrix is tiled out anyway (for gemm at least) so column vs row ordering is probably a little less important nowadays (which isn’t to say it never matters).

Re: Matrices can be your friends (2002)

#56

Post by ndriscoll »

kccqzy wrote: post #47
getnormality wrote: post #6

People must get taught math terribly if they think "I don't need to worry about piles of abstract math to understand a rotation, all I have to do is think about what happens to the XYZ axes under the matrix rotation". That is what you should learn in the math class! Anyone who has taken linear algebra should know that (1) a rotation is a linear operation, (2) the result of a linear operation is calculated with matrix…

I have taken several linear algebra courses, one from my high school and two from universities. The thing is, not all courses of linear algebra will discuss rotations the way you discuss it. One reason is that sometimes a high school linear algebra course cannot assume students have learned trigonometry. I've seen teachers teach it just to solve larger linear systems of equations. Another reason is that sometimes a c…

Usually you just draw a 2D or 3D picture and say "n" while pointing to it. e.g. I had a professor that drew a 2D picture on a board where he labeled one axis R^m and the other R^n and then drew a "graph" when discussing something like the implicit function theorem. One takeaway of a lot of linear algebra is that doing this is more-or-less correct (and then functional analysis tells you this is still kind-of correct-ish even in infinite dimensions). Actually SVD tells you in some sense that if you look at things in the right way, the "heart" of a linear map is just 1D multiplication acting independently along different axes, so you don't need to consider all n dimensions at once.

Re: Matrices can be your friends (2002)

#57

Post by jesuslop »

yoda222 wrote: post #27
Viliam1234 wrote: post #5

> Mathematicians like to see their matrices laid out on paper this way (with the array indices increasing down the columns instead of across the rows as a programmer would usually write them). Could a mathematician please confirm of disconfirm this? I think that different branches of mathematics have different rules about this, which is why careful writers make it explicit.

Not a mathematician, just an engineer that used matrix a lot (and even worked for MathWorks at one point), I would say that most mathematicians don't care. Matrix is 2D, they don't have a good way to be laid out in 1D (which is what is done here, by giving them linear indices). They should not be represented in 1D. The only type of mathematicians that actually care are: - the one that use software where using one or…

ah, so you can get row vectors with a type cast in C but not column ones. While in Fortran and friends is the converse (if they casted). Yep that is more mathy. Linear map evaluation is linear combination of columns.

EDIT: type cast or just single square bracket application in C

Re: Matrices can be your friends (2002)

#58

Post by baruz »

getnormality wrote: post #6

People must get taught math terribly if they think "I don't need to worry about piles of abstract math to understand a rotation, all I have to do is think about what happens to the XYZ axes under the matrix rotation". That is what you should learn in the math class! Anyone who has taken linear algebra should know that (1) a rotation is a linear operation, (2) the result of a linear operation is calculated with matrix…

This was part of Steve Baker’s (“Omniverous Hexapod”, sic) extensions to a long-standing Usenet FAQ about graphics programming, put out by “Carniverous Hexapod” (sic). It’s at least two decades old, and the FAQ from which he it on may be from the 1990s? I have the niggling recollection that the Carniverous name may have been based on Vernor Vinge’s _Fire upon the deep_ aliens.

He did not invent it, but he probably had to deal with aspiring graphics programmers who were not very math-savvy.

Re: Matrices can be your friends (2002)

#59

Post by zaik »

Viliam1234 wrote: post #5

> Mathematicians like to see their matrices laid out on paper this way (with the array indices increasing down the columns instead of across the rows as a programmer would usually write them). Could a mathematician please confirm of disconfirm this? I think that different branches of mathematics have different rules about this, which is why careful writers make it explicit.

Recently graduated math student here. The definition of the "vec" operator which turns a matrix into a vector works like this, stacking up columns rather than rows.

https://en.wikipedia.org/wiki/Vectorization_(mathematics)

Re: Matrices can be your friends (2002)

#60

Post by motorest »

dgacmu wrote: post #17
zelphirkalt wrote: post #15

Earlier quoted context omitted.

When I was studying and made the mistake of choosing 3D computer graphics as a lecture, I remember some 4x4 matrix that was used for rotation, with all kinds of weird terms in it, derived only once, in a way I was not able to understand and that didn't relate to any visual idea or imagination, which makes it extra hard for me to understand it, because I rely a lot on visualization of everything. So basically, there w…

You can do rotation with a 3x3 matrix. The first lecture was using a 4x4 matrix because you can use it for a more general set of transformations, including affine transforms (think: translating an object by moving it in a particular direction). Since you can combine a series of matrix multiplications by just pre-multiplying the matrix, this sets you up for doing a very efficient "move, scale, rotate" of an object usi…

> The first lecture was using a 4x4 matrix because you can use it for a more general set of transformations, including affine transforms (think: translating an object by moving it in a particular direction).

I think this is mixing up concepts that are orthogonal to linear spaces, linear transformations, and even specific operations such as rotations.

The way you mention "more general set of transformations" suggests you're actually referring to homogeneous coordinates, which is a trick that allows a subset of matrix-vector multiplication and vector addition in 3D spaces to be expressed as a single matrix-vector multiplication in 4D space.

This is fine and dandy if your goal is to take a headstart to 3D programming, where APIs are already designed around this. This is however a constrained level of abstraction above actual linear algebra, which may be and often is more confusing.

Post reply on HN