The computations in transformers are actually generalized tensor tensor contractions implemented as matrix multiplications. Their efficient implementation in gpu hardware involves many algebraic gems and is a work of art. You can have a taste of the complexity involved in their design in this Youtube video: https://www.youtube.com/live/ufa4pmBOBT8
Is Matrix Multiplication Ugly?
41–50 of 113 posts
Re: Is Matrix Multiplication Ugly?
#42Earlier quoted context omitted.
the aesthetics of math and physics is by far the most boring discussion that can be had. i used to be utterly repulsed by such talk in undergrad - beauty this and that. it absolutely always felt affected and put on - as if you talk about it enough, you'll actually convince people outside of the major to give you the same plaudits as real artists.... yea right lol.
That’s the opinion of one person. There are many mathematicians who find beauty in the maths they are studying.
You're equivocating over the verdict, here. Are they right?
Re: Is Matrix Multiplication Ugly?
#43Earlier quoted context omitted.
> When I was in college they told us most of computing resources were used doing Linear Programming. I seriously doubt that was ever true, except perhaps for a very brief time in the 1950s or 60s. Linear programming is an incredibly niche application of computing used so infrequently that I've never seen it utilised anywhere despite being a consultant that has visited hundreds of varied customers including big busine…
The first two results from Google with "Voronoi astro" gave two different uses than the one I knew about (sampling fibre bundles): https://galaxyproject.org/news/2025-06-11-voronoi-astronomy/ https://arxiv.org/abs/2511.14697
Astronomy is pure research and is performed almost exclusively by academics.
I’m not saying these things have zero utility, it’s just that they’re used far less frequently in industry than academics imagine.
Re: Is Matrix Multiplication Ugly?
#44Earlier quoted context omitted.
the aesthetics of math and physics is by far the most boring discussion that can be had. i used to be utterly repulsed by such talk in undergrad - beauty this and that. it absolutely always felt affected and put on - as if you talk about it enough, you'll actually convince people outside of the major to give you the same plaudits as real artists.... yea right lol.
That’s the opinion of one person. There are many mathematicians who find beauty in the maths they are studying.
> There are many mathematicians who find beauty
Lol did you think this was clever? You just literally reiterated exactly what I said. See, if you had said "there are many pianists that find beauty in math" - you know like how many mathematicians find beauty in piano concertos - then you'd have me.
Re: Is Matrix Multiplication Ugly?
#45I doubt anyone of the past or present could fully describe what a matrix is, and what its multiplication is. There are many ways people looked at it so far - as a spatial transformation, dot products and so on. I don't think the description is complete in any significant way.
That's because we don't fully understand what a number is and what a multiplication is. We defined -x and 1/x as inverses (additive and multiplicative), but what is -1/x ? Let's consider them as operations. Apply any one of them on any other of them, you get the third one. Thus they occupy peer status. But we hardly ever talked about -1/x.
The mathematical inquisition is in its infancy.
Re: Is Matrix Multiplication Ugly?
#46I doubt anyone of the past or present could fully describe what a matrix is, and what its multiplication is. There are many ways people looked at it so far - as a spatial transformation, dot products and so on. I don't think the description is complete in any significant way. That's because we don't fully understand what a number is and what a multiplication is. We defined -x and 1/x as inverses (additive and multipl…
I get your point, but i think the real issue is -(1/(-1/x)). It is the one that is being overlooked the most in our society, as if it were something normal, but it contains some of the deepest truths imho.
Re: Is Matrix Multiplication Ugly?
#47I doubt anyone of the past or present could fully describe what a matrix is, and what its multiplication is. There are many ways people looked at it so far - as a spatial transformation, dot products and so on. I don't think the description is complete in any significant way. That's because we don't fully understand what a number is and what a multiplication is. We defined -x and 1/x as inverses (additive and multipl…
I get your point, but i think the real issue is -(1/(-1/x)). It is the one that is being overlooked the most in our society, as if it were something normal, but it contains some of the deepest truths imho.
how about -1/(-(1/(-1/x))) ? How many roads must a man walk down before we can call him a man ?
Re: Is Matrix Multiplication Ugly?
#48I doubt anyone of the past or present could fully describe what a matrix is, and what its multiplication is. There are many ways people looked at it so far - as a spatial transformation, dot products and so on. I don't think the description is complete in any significant way. That's because we don't fully understand what a number is and what a multiplication is. We defined -x and 1/x as inverses (additive and multipl…
As someone who never got deeply into math but deeply into programming they just seemed like an incompletely generalized data structure with an interesting "canonical" algorithm that can be used on it. In some cases, if you arrange your data into the structure correctly, you can use it to model interesting real world phenomenon.
It feels like Linear Algebra tries to get at the heart of this generality but the structure and operator is more constrained than it ultimately could be. It's a small oddball computational device that can be tersely written into papers and widely understood. I always find pseudocode easier to follow and reason about but that's my particular bias.
Re: Is Matrix Multiplication Ugly?
#49Do you disagree with my take or think I’m missing Witt’s point? I’d be happy to hear from people who disagree with me.
I think it conflates the map and the territory.
Linear transformations are a beautiful thing, but matrices are an ugly representation that nevertheless is a convenient one when we actually want to compute.
Elegant territory. Inelegant, brute-force, number crunching map.