Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Matrix Calculus

matrixcalculus.org

Post reply on HN
Print view View original

41–50 of 53 posts

Re: Matrix Calculus

#41

Post by SoerenL »

jkam wrote: post #39
a1k0n wrote: post #37

Would be nice if there was a "symmetric matrix" variable type, so it could simplify Ax + A'x to 2Ax if A is symmetric.

I wonder why they don't do it. In the tool I had running, we handled this by removing any transpose of a symmetric matrix (after propagating it before the leaves). Together with the simplification rule x + x -> 2*x for any x, you get the expected result. I could only guess why they didn't include it in the online matrix calculus tool. It was published after I left the group.

It is not in the current online tool but we will add it again soon. It is still in there the way you describe it (passing transpose down to the leaves and simplification rules as well). Btw: How are you doing and where have you been? Would be nice to also add a link to you and your current site. GENO is also on its way.

Re: Matrix Calculus

#42

Post by scentoni »

mindcrime wrote: post #38
surajx wrote: post #11

Earlier quoted context omitted.

The best resource I have come across is the Wikipedia page. It's comprehensive and concise. https://en.wikipedia.org/wiki/Matrix_calculus

This kind of stuff just infuriates me. I can't see any good reason for having so many competing notational standards for the same thing: The notation used here is commonly used in statistics and engineering, while the tensor index notation is preferred in physics. Two competing notational conventions split the field of matrix calculus into two separate groups. The two groups can be distinguished by whether they write…

Some other examples:

sign conventions in thermodynamics

conventions in Fourier transforms

short billion vs long billion

calorie vs Calorie

English vs. metric units

SI vs cgs metric

esu vs Gaussian units in EM

Re: Matrix Calculus

#43

Post by qubex »

mjfl wrote: post #4

Along the same lines, does there exist an "algebra checker" that could, say, take in two successive lines of latex, with perhaps a hint of how to get from one to the other, and confirm that there are no algebra errors?

On a whim I programmed a symbolic integral-solving library in Python (I am not a professional programmer) and, as an ill-advised ’extension’, bolted on the ability to receive and emit LATEX expressions. Long story short, I was boasting about it on Nichols Nassim Taleb’s twitter feed, submitted an answer to a problem question he had posed that got mis-parsed by the front end, gave the wrong answer (which was entirely consistent, because the check stage compared the answer by deriving the flawed question), and got banned, flamed, and persecuted to Hell and back. True story.

From this I learnt the hard way that tokenisation and representation (in LATEX or MathML) do not belong in the same place as a CAS (Computer-assisted Algebra System).

Re: Matrix Calculus

#44

Post by qubex »

yorwba wrote: post #8
CodesInChaos wrote: post #7

Earlier quoted context omitted.

The choice of LaTeX comes with two problems: 1) It's designed for describe graphical representation and not semantics. 2) parsing it properly is very difficult, especially considering how macros and other definitions can influence the syntax.

You could: 1) assume a standard interpretation of symbols (e.g. this site uses a-g for scalars, h-z for vectors and A-Z for matrices) and make those assumptions explicit and modifiable (e.g. WolframAlpha displays "assuming X is a Y, use as a Z instead"). 2) Support only the subset that MathJax can handle, which seems to be enough for most purposes. I definitely agree that LaTeX is not the optimal input format for tha…

See my comment above about my experiences with this: I tried, and a corner case killed me. There’s a reason why Mathematica and other CAS packages have invented their own idiosyncratic notations. LATEX and MathML are truly hopeless when it comes to communicating with algebra systems. It looks deceptively ’doable’, but really it’s a recipe for disaster.

Re: Matrix Calculus

#45

Post by SoerenL »

mjfl wrote: post #4

Along the same lines, does there exist an "algebra checker" that could, say, take in two successive lines of latex, with perhaps a hint of how to get from one to the other, and confirm that there are no algebra errors?

As one of the authors of this tool I understand that you would like to have some way of trusting the output.

Internally we check it numerically, i.e., generate some random data for the given variables and check the derivative by comparing it to an approximation via finite differences. We will ship this code (hopefully soon) with one of the next versions. You can then check it yourself. Otherwise, as far as I know there does not exist any other tool that computes matrix derivatives so I understand it is hard to convince anyone of the correctness of the results. But I hope the numerical tests will be helpful.

Re: Matrix Calculus

#46

Post by yorwba »

qubex wrote: post #43
mjfl wrote: post #4

Along the same lines, does there exist an "algebra checker" that could, say, take in two successive lines of latex, with perhaps a hint of how to get from one to the other, and confirm that there are no algebra errors?

On a whim I programmed a symbolic integral-solving library in Python (I am not a professional programmer) and, as an ill-advised ’extension’, bolted on the ability to receive and emit LATEX expressions. Long story short, I was boasting about it on Nichols Nassim Taleb’s twitter feed, submitted an answer to a problem question he had posed that got mis-parsed by the front end, gave the wrong answer (which was entirely…

I believe that LaTeX/MathML absolutely do belong in a CAS, just not as the primary means of interaction. (I.e. you do your work in the CAS's own language and then export to LaTeX.)

But maybe your example can convince me otherwise. Could you show the specific LaTeX code in question and describe how your program mishandled it?

Re: Matrix Calculus

#47

Post by jkam »

SoerenL wrote: post #41
jkam wrote: post #39

Earlier quoted context omitted.

I wonder why they don't do it. In the tool I had running, we handled this by removing any transpose of a symmetric matrix (after propagating it before the leaves). Together with the simplification rule x + x -> 2*x for any x, you get the expected result. I could only guess why they didn't include it in the online matrix calculus tool. It was published after I left the group.

It is not in the current online tool but we will add it again soon. It is still in there the way you describe it (passing transpose down to the leaves and simplification rules as well). Btw: How are you doing and where have you been? Would be nice to also add a link to you and your current site. GENO is also on its way.

I'm good. Looking at things from the data angle now. But unfortunately no public page. You can link to the old one, if you want to. Have you compared against TensorFlow XLA?

Re: Matrix Calculus

#48

Post by qubex »

yorwba wrote: post #46
qubex wrote: post #43

Earlier quoted context omitted.

On a whim I programmed a symbolic integral-solving library in Python (I am not a professional programmer) and, as an ill-advised ’extension’, bolted on the ability to receive and emit LATEX expressions. Long story short, I was boasting about it on Nichols Nassim Taleb’s twitter feed, submitted an answer to a problem question he had posed that got mis-parsed by the front end, gave the wrong answer (which was entirely…

I believe that LaTeX/MathML absolutely do belong in a CAS, just not as the primary means of interaction. (I.e. you do your work in the CAS's own language and then export to LaTeX.) But maybe your example can convince me otherwise. Could you show the specific LaTeX code in question and describe how your program mishandled it?

I had set myself the target of implementing the Risch Algorithm (https://en.wikipedia.org/wiki/Risch_algorithm) in SymPy (3) but the event was so embarrassing I deleted my code and never published it publicly.

I’m not a programmer. I erred in trying to do a programmer’s job. Also, I discovered why Nicholas Nassim Taleb has the reputation for being rather uncharitable (but deep down I feel like I deserved it, because hey, I stated a mathematical untruth).

Re: Matrix Calculus

#49

Post by SoerenL »

jkam wrote: post #47
SoerenL wrote: post #41

Earlier quoted context omitted.

It is not in the current online tool but we will add it again soon. It is still in there the way you describe it (passing transpose down to the leaves and simplification rules as well). Btw: How are you doing and where have you been? Would be nice to also add a link to you and your current site. GENO is also on its way.

I'm good. Looking at things from the data angle now. But unfortunately no public page. You can link to the old one, if you want to. Have you compared against TensorFlow XLA?

I did not compare to Tensorflow XLA but I compared it to Tensorflow. Of course, it depends on the problem. For instance, for evaluating the Hessian of x'Ax MC is a factor of 100 faster than TF. But MC and TF have different objectives. TF more on scalar valued functions as needed for deep learning, MC for the general case, especially also vector and matrix valued functions as needed for dealing with constraints. But I will give TF XLA a try.

Re: Matrix Calculus

#50

Post by digitalshankar »

jey wrote: post #34
platz wrote: post #31

Earlier quoted context omitted.

How did you start in machine learning. Did you come from a top 10 school?

No, I failed out of university and am self-taught. I spent many years deliberately underemployed working on self-study and coding up projects at the limit of my abilities. I had decided to bet on investing in myself instead of just carrying out some boss’s wishes, which would be more optimized to benefit the company than my own growth and development. Here’s a great resource if you’re starting out today: http://datas…

Inspiring story!!
Post reply on HN