Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Matrix Multiplication Ugly?

mathenchant.wordpress.com

Post reply on HN
Print view View original

51–60 of 113 posts

Re: Is Matrix Multiplication Ugly?

#51

Post by zkmon »

youoy wrote: post #46
zkmon wrote: post #45

I doubt anyone of the past or present could fully describe what a matrix is, and what its multiplication is. There are many ways people looked at it so far - as a spatial transformation, dot products and so on. I don't think the description is complete in any significant way. That's because we don't fully understand what a number is and what a multiplication is. We defined -x and 1/x as inverses (additive and multipl…

I get your point, but i think the real issue is -(1/(-1/x)). It is the one that is being overlooked the most in our society, as if it were something normal, but it contains some of the deepest truths imho.

No you didn't get it. You missed "Let's consider them as operations. Apply any one of them on any other of them, you get the third one."

Re: Is Matrix Multiplication Ugly?

#52

Post by zkmon »

iamgopal wrote: post #47
youoy wrote: post #46

Earlier quoted context omitted.

I get your point, but i think the real issue is -(1/(-1/x)). It is the one that is being overlooked the most in our society, as if it were something normal, but it contains some of the deepest truths imho.

how about -1/(-(1/(-1/x))) ? How many roads must a man walk down before we can call him a man ?

No need of walking, they just need to be able to read post properly before calling him a man.

Re: Is Matrix Multiplication Ugly?

#53

Post by nialse »

hobs wrote: post #17
sfpotter wrote: post #4

I think this sentence: > But matrix multiplication, to which our civilization is now devoting so many of its marginal resources, has all the elegance of a man hammering a nail into a board. is the most interesting one. A man hammering a nail into a board can be both beautiful and elegant! If you've ever seen someone effortlessly hammer nail after nail into wood without having to think hardly at all about what they're…

https://www.youtube.com/watch?v=Ruf-cLr2PZ8 I always think of this when thinking about the gracefulness of a hammer.

Turns out it’s the skill of the person handling the hammer that matters most. Enlightening! Appreciate the link!

Re: Is Matrix Multiplication Ugly?

#55

Post by znkr »

algernonramone wrote: post #32

I am willing to admit that I find matrix multiplication ugly, as well as non-intuitive. But, I am also willing to admit that my finding it ugly is likely a result of my relative mathematical immaturity (despite my BS in math).

Maybe it helps to think of matrix multiplication as a special case of the composition of linear transformation. In the finite dimensional case they can be expressed as matrix multiplications.

Re: Is Matrix Multiplication Ugly?

#56

Post by countWSS »

Beauty,symmetry,etc are largely irrelevant, the key point it does not scale and burning gigawatts to compute these matrices(even with all those tricks) will not scale or compete with more efficient/direct methods in the long term. Perhaps transformers are very elaborate sunk-cost fallacy where pivoting to scalable, simpler architecture is treated as "too risky" even when cost of new GPU cluster dwarfs whatever it takes to bring an architecture from 0 to chatGPT level.

Re: Is Matrix Multiplication Ugly?

#57

Post by sigmoid10 »

countWSS wrote: post #56

Beauty,symmetry,etc are largely irrelevant, the key point it does not scale and burning gigawatts to compute these matrices(even with all those tricks) will not scale or compete with more efficient/direct methods in the long term. Perhaps transformers are very elaborate sunk-cost fallacy where pivoting to scalable, simpler architecture is treated as "too risky" even when cost of new GPU cluster dwarfs whatever it tak…

The whole issue with this industry is that it moves so fast, there is no "long term." You're either in all the way in a likely futile attempt to capture the market or you're not in at all. So you also don't have time to really innovate on the hardware or software level and you need to put everything into training data and training hardware.

Re: Is Matrix Multiplication Ugly?

#58

Post by youoy »

zkmon wrote: post #51
youoy wrote: post #46

Earlier quoted context omitted.

I get your point, but i think the real issue is -(1/(-1/x)). It is the one that is being overlooked the most in our society, as if it were something normal, but it contains some of the deepest truths imho.

No you didn't get it. You missed " Let's consider them as operations. Apply any one of them on any other of them, you get the third one. "

So is what i wrote a third one? Fourth? Fifth? :)

Re: Is Matrix Multiplication Ugly?

#60

Post by zkmon »

youoy wrote: post #58
zkmon wrote: post #51

Earlier quoted context omitted.

No you didn't get it. You missed " Let's consider them as operations. Apply any one of them on any other of them, you get the third one. "

So is what i wrote a third one? Fourth? Fifth? :)

Not sure what you are talking about. What you wrote reduces to just x. What I meant was, if you substitute say, -x for x in -1/x, you get 1/x, which is the third inverse. Same is true for the other two pairs. So, if we call them functions f, g and h, then, f=g(h)=h(g); g=f(h)=h(f); h=f(g)=g(f)
Post reply on HN