Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Matrix Multiplication Ugly?

mathenchant.wordpress.com

Post reply on HN
Print view View original

101–110 of 113 posts

Re: Is Matrix Multiplication Ugly?

#101

Post by jiggawatts »

srean wrote: post #68
jiggawatts wrote: post #20

Earlier quoted context omitted.

> When I was in college they told us most of computing resources were used doing Linear Programming. I seriously doubt that was ever true, except perhaps for a very brief time in the 1950s or 60s. Linear programming is an incredibly niche application of computing used so infrequently that I've never seen it utilised anywhere despite being a consultant that has visited hundreds of varied customers including big busine…

3d modelers would like to have a word with you. Part of the reason why linear programming does need t get used as often is that there are no industry standard software implementation that is not atrociously priced. Same deal with Mathematica.

3D modelling is mostly linear algebra, not linear programming, which is an entirely different set of algorithms.

Re: Is Matrix Multiplication Ugly?

#102

Post by jiggawatts »

petters wrote: post #76
jiggawatts wrote: post #20

Earlier quoted context omitted.

> When I was in college they told us most of computing resources were used doing Linear Programming. I seriously doubt that was ever true, except perhaps for a very brief time in the 1950s or 60s. Linear programming is an incredibly niche application of computing used so infrequently that I've never seen it utilised anywhere despite being a consultant that has visited hundreds of varied customers including big busine…

I have not only used linear programming in the industry, I have also had to write my own solver because the existing ones (even commercial) were to slow. (This was possible only because I only cares about a very approximate solution) The triangulations you mention are important in the current group I'm working in.

I'm curious to hear what you specifically use these algorithms for!

PS: My point is not that these things are never used, they clearly are, I'm saying that the majority of CPU cycles globally goes towards "idle", then pushing pixels around with simple bitblt-like algorithms for 2D graphics, then whatever it is that browsers do on the inside, then operating system internals, and then specialised and more interesting algorithms like Linear Programming are a vanishingly small slice of whatever is left of that pie chart.

Re: Is Matrix Multiplication Ugly?

#103

Post by woopwoop »

Matrix multiplication is not ugly, but matrices themselves are ugly, mainly because they encode the arbitrary operation of choosing a basis. There's nothing especially nice about the pixel basis for images, or about the token basis for language. But of all the things that make up modern deep learning, matrix multiplication is surely the _least_ ugly. Relu/gelu is not pretty! Batch normalization is vomit-inducing!! Imagenet normalization? JFC!!!

Re: Is Matrix Multiplication Ugly?

#105

Post by srean »

jiggawatts wrote: post #101
srean wrote: post #68

Earlier quoted context omitted.

3d modelers would like to have a word with you. Part of the reason why linear programming does need t get used as often is that there are no industry standard software implementation that is not atrociously priced. Same deal with Mathematica.

3D modelling is mostly linear algebra , not linear programming , which is an entirely different set of algorithms.

Oh I mentioned it in the context of mesh geometries, tesselations.

Re: Is Matrix Multiplication Ugly?

#106

Post by srean »

almostgotcaught wrote: post #96
srean wrote: post #91

Earlier quoted context omitted.

Yup. PhD is a start. My condolences though, for being in a line of work where you don't perceive beauty.

I'm a researcher in FAANG - I'll keep in mind your condolences when I get my yearly RSU re-grant lololol.

Then thanks for your services, your work is literally paying me in my retirement.

The 15 is on the lower side. When I used to be there 15 would be on the uncomfortable side :) Good luck to up the numbers. Oh! do get back on the Cooley Tukey citations and FFT mention ratio.

Re: Is Matrix Multiplication Ugly?

#107

Post by bee_rider »

btilly wrote: post #83
laichzeit0 wrote: post #29

Well function composition f(g(x)) is not the same as g(f(x)) and when you represent f and g as matrices relative to some suitable set of basis functions then obviously AB and BA should be different. If the multiplication was defined any different, that wouldn’t work.

The way that I used to put this was, "If I put on my shoes before my socks, I'll get a different result than if it I put on my socks before my shoes. Order of operations matters."

The ticket collector asked me why I was getting dressed for work on the train.

I asked him, “is this not the commuter rail?”

Re: Is Matrix Multiplication Ugly?

#108

Post by almostgotcaught »

srean wrote: post #106
almostgotcaught wrote: post #96

Earlier quoted context omitted.

I'm a researcher in FAANG - I'll keep in mind your condolences when I get my yearly RSU re-grant lololol.

Then thanks for your services, your work is literally paying me in my retirement. The 15 is on the lower side. When I used to be there 15 would be on the uncomfortable side :) Good luck to up the numbers. Oh! do get back on the Cooley Tukey citations and FFT mention ratio.

You know you have a web presence right? You know it's very easy to verify you've never been in FAANG right? lolol

Re: Is Matrix Multiplication Ugly?

#109

Post by 1718627440 »

almostgotcaught wrote: post #86
srean wrote: post #84

Earlier quoted context omitted.

> You can still find citations of those papers to this day. That's not what I contested. What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point. You can count number of such citations in last 10 years of say neural nets literature, or applied maths literature and report. Thats plenty of use. Citations to their differential calculus that are still made are mos…

> What fraction of people who use differentials in their published work still cite Newton or Leibnitz was the point Those papers were written in the 1600s. "The character of physical law", the essay you're ripping off, was written in 1964. 100% papers from the 1960s are cited every single time the techniques are used. You are as tedious as the original refrain I was complaining about (which is not at all ironic). Wha…

And an HN comment is not a scientific paper. When I tell people about ideas I find insightful, I don't cite a proper source either. If they like the idea I might tell them later, how I got it or where they can find more detail about the idea.

Honestly quite often an idea did originate in my own thoughts, but the work to put it into well-formed words, which I will use to tell others about it, was done by someone else, whose formulations of the same idea I had, I have read later.

Post reply on HN