Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Matrix Multiplication Ugly?

mathenchant.wordpress.com

Post reply on HN
Print view View original

81–90 of 113 posts

Re: Is Matrix Multiplication Ugly?

#81

Post by srean »

almostgotcaught wrote: post #80
srean wrote: post #79

Earlier quoted context omitted.

Quite the contrary. I expect majority of HN readers to know the quote, base 0 if you will, and not harbor thoughts that by having read it they are a part of an exclusive club. Channeling Good Will Hunting much huh? Most HN'ers would have watched that too.

I have no idea what you're trying to say - it is generally understood everywhere in the world (ie all forms of human culture) that it's pathetic to pass off someone else's insights as your own.

Only when there is an expectation of being perceived as original. When people use differentiation people don't cite Newton or Leibnitz, do they ?

Re: Is Matrix Multiplication Ugly?

#82

Post by almostgotcaught »

srean wrote: post #81
almostgotcaught wrote: post #80

Earlier quoted context omitted.

I have no idea what you're trying to say - it is generally understood everywhere in the world (ie all forms of human culture) that it's pathetic to pass off someone else's insights as your own.

Only when there is an expectation of being perceived as original. When people use differentiation people don't cite Newton or Leibnitz, do they ?

.... Of course they do? You can still find citations of those papers to this day.

For all of your "forceful" comments on math, I think probably you don't actually know much about it.

Re: Is Matrix Multiplication Ugly?

#83

Post by btilly »

laichzeit0 wrote: post #29

Well function composition f(g(x)) is not the same as g(f(x)) and when you represent f and g as matrices relative to some suitable set of basis functions then obviously AB and BA should be different. If the multiplication was defined any different, that wouldn’t work.

The way that I used to put this was, "If I put on my shoes before my socks, I'll get a different result than if it I put on my socks before my shoes. Order of operations matters."

Re: Is Matrix Multiplication Ugly?

#84

Post by srean »

almostgotcaught wrote: post #82
srean wrote: post #81

Earlier quoted context omitted.

Only when there is an expectation of being perceived as original. When people use differentiation people don't cite Newton or Leibnitz, do they ?

.... Of course they do? You can still find citations of those papers to this day. For all of your "forceful" comments on math, I think probably you don't actually know much about it.

> You can still find citations of those papers to this day.

That's not what I contested. What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point. You can count number of such citations in last 10 years of say neural nets literature, or applied maths literature and report. Thats plenty of use.

Citations to their differential calculus that are still made are mostly in the context of history of math.

Seems numeracy or comprehension is not your strong point. LOL.

> I have no idea what you're trying to say

Now I don't doubt it. LOL

Re: Is Matrix Multiplication Ugly?

#85

Post by peterfirefly »

mhb wrote: post #73
peterfirefly wrote: post #61

I just finished reading lots of Stephen Witt quotes on goodreads. He comes across as a white Malcolm Gladwell, except that he actually does know what "Igon values" are so I don't know what his excuse is.

> white Malcolm Gladwell I'm intrigued. How would a white Malcolm Gladwell's quotes differ from the IRL Malcolm Gladwell?

Different hair, mostly.

Re: Is Matrix Multiplication Ugly?

#86

Post by almostgotcaught »

srean wrote: post #84
almostgotcaught wrote: post #82

Earlier quoted context omitted.

.... Of course they do? You can still find citations of those papers to this day. For all of your "forceful" comments on math, I think probably you don't actually know much about it.

> You can still find citations of those papers to this day. That's not what I contested. What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point. You can count number of such citations in last 10 years of say neural nets literature, or applied maths literature and report. Thats plenty of use. Citations to their differential calculus that are still made are mos…

> What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point

Those papers were written in the 1600s. "The character of physical law", the essay you're ripping off, was written in 1964. 100% papers from the 1960s are cited every single time the techniques are used.

You are as tedious as the original refrain I was complaining about (which is not at all ironic). What's most tedious is you're not actually a mathematician but presume to speak for them.

Re: Is Matrix Multiplication Ugly?

#87

Post by ComplexSystems »

Matrices represent linear transformations. Linear transformations are very natural and "beautiful" things. They are also very clearly not commutative: f(g(x)) is not the same as g(f(x)). The matrix algebra perfectly represents all of this, and as a result, FGx is not the same as GFx. It's only not "beautiful" if you believe that matrix multiplication is a random operation that exists for no reason.

Re: Is Matrix Multiplication Ugly?

#88

Post by btown »

sdenton4 wrote: post #30
sfpotter wrote: post #4

I think this sentence: > But matrix multiplication, to which our civilization is now devoting so many of its marginal resources, has all the elegance of a man hammering a nail into a board. is the most interesting one. A man hammering a nail into a board can be both beautiful and elegant! If you've ever seen someone effortlessly hammer nail after nail into wood without having to think hardly at all about what they're…

A somewhat more beautiful matmul for neural networks is given by the Monarch paper: https://arxiv.org/abs/2204.00595 Generally, low-rank and block-diagonal matrices are both great strategies for producing expressive matmuls with fewer parameters. We can view the FFT as a particularly deft example of factorizing one big matmul into a number of block-diagonal matmuls, greatly reducing the overall number of multiplicati…

https://hazyresearch.stanford.edu/blog/2023-12-11-truly-subq... is an approachable overview of the Monarch approach, for those interested!

There's a lot of opportunity here. Just because matrix multiplication makes for a beautiful mathematical building block, and a very reasonable one to build high-level ML logic on, doesn't mean it needs to be computed the same way, and in the same order, that we learned in linear algebra courses.

I'm quite curious if this is being used in practice at scale, or whether it's still in the lab at the moment!

Re: Is Matrix Multiplication Ugly?

#89

Post by srean »

almostgotcaught wrote: post #86
srean wrote: post #84

Earlier quoted context omitted.

> You can still find citations of those papers to this day. That's not what I contested. What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point. You can count number of such citations in last 10 years of say neural nets literature, or applied maths literature and report. Thats plenty of use. Citations to their differential calculus that are still made are mos…

> What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point Those papers were written in the 1600s. "The character of physical law", the essay you're ripping off, was written in 1964. 100% papers from the 1960s are cited every single time the techniques are used. You are as tedious as the original refrain I was complaining about (which is not at all ironic). Wha…

You are changing goal posts now. Your absolute claim was

> it's pathetic to pass off someone else's insights as your own.

To which my point was citations are made when there is an expectation of originality. By now Feynman's anecdotes are folklore and folks wisdom.

OK let's go by your standards. Cooley Tukey's FFT algorithm was "discovered" by them in around 1965. How often do they get a citation when FFT is used, especially in comments on a social site, such as HN is.

LOL even 10 years old results do not get cited because they are considered common knowledge.

That said, Witt's notion of beauty that Propp is critiquing in the posted article is just plane idiotic. Lack of commutativity is not lack of beauty. What a stupid idea.

Mathematical beauty and imagination is different. One of Hilbert's grad students dropped out to become a poet.Hilbert is reported to have said: 'I never thought he had enough imagination to be a mathematician.'

A little unsolicited advice: if you are an aspiring mathematician(I am very happy for you if you are), but if you do not have a sense of a good taste or mathematical beauty, you probably will probably not have a good time.

Re: Is Matrix Multiplication Ugly?

#90

Post by almostgotcaught »

srean wrote: post #89
almostgotcaught wrote: post #86

Earlier quoted context omitted.

> What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point Those papers were written in the 1600s. "The character of physical law", the essay you're ripping off, was written in 1964. 100% papers from the 1960s are cited every single time the techniques are used. You are as tedious as the original refrain I was complaining about (which is not at all ironic). Wha…

You are changing goal posts now. Your absolute claim was > it's pathetic to pass off someone else's insights as your own. To which my point was citations are made when there is an expectation of originality. By now Feynman's anecdotes are folklore and folks wisdom. OK let's go by your standards. Cooley Tukey's FFT algorithm was "discovered" by them in around 1965. How often do they get a citation when FFT is used, es…

> if you are an aspiring mathematician, and you do not have a sense of a good taste or mathematical beauty, you probably will probably not have a good time

Lol I have a PhD from a T10 and 15 published papers. I'm pretty sure I don't need your advice on "taste" or "beauty".

Post reply on HN