Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Matrix Calculus

matrixcalculus.org

Post reply on HN
Print view View original

21–30 of 53 posts

Re: Matrix Calculus

#21

Post by rav »

cosmic_ape wrote: post #14
mjfl wrote: post #4

Along the same lines, does there exist an "algebra checker" that could, say, take in two successive lines of latex, with perhaps a hint of how to get from one to the other, and confirm that there are no algebra errors?

SymPy does not take Latex as input, but it has symbolic equivalence checkers. Methods are highly heuristic, and some languages (all expressions with,say, \pi, and exp function, or something similar) are in general undecidable.

To elaborate: Symbolic equivalence checking for two expressions f1 and f2 can be done by seeing if sympy.simplify(sympy.Eq(f1, f2)) is sympy.true.

  import sympy
  from sympy.abc import x, y, alpha, s
  quad = s ** 2 - alpha * s - 2
  # Let s1 and s2 be the two solutions to the quadratic equation 'quad == 0'
  s1, s2 = sympy.solve(quad, s)
  u = (x - s2) / (x - s1) * (y - s1) / (y - s2)
  f1 = (s2 - s1 * u) / (1 - u)
  f2 = (x * y - alpha * x - 2) / (y - x)
  # Claim: f1 is equal to f2
  print(sympy.simplify(sympy.Eq(f1, f2)))
  # Prints "True"

Re: Matrix Calculus

#23

Post by yorwba »

rav wrote: post #21
cosmic_ape wrote: post #14

Earlier quoted context omitted.

SymPy does not take Latex as input, but it has symbolic equivalence checkers. Methods are highly heuristic, and some languages (all expressions with,say, \pi, and exp function, or something similar) are in general undecidable.

To elaborate: Symbolic equivalence checking for two expressions f1 and f2 can be done by seeing if sympy.simplify(sympy.Eq(f1, f2)) is sympy.true. import sympy from sympy.abc import x, y, alpha, s quad = s ** 2 - alpha * s - 2 # Let s1 and s2 be the two solutions to the quadratic equation 'quad == 0' s1, s2 = sympy.solve(quad, s) u = (x - s2) / (x - s1) * (y - s1) / (y - s2) f1 = (s2 - s1 * u) / (1 - u) f2 = (x * y -…

Is there something for implication or equivalence of equations? I was trying the example from Smaug123's sibling comment (https://news.ycombinator.com/item?id=15728598), but

  >>> import sympy as sp
  >>> x = sp.Symbol('x')
  >>> sp.simplify(sp.Implies(sp.Eq(x**2 + 2*x + 1, 0), sp.Eq(x, -1)))

  Eq(x, -1) | Ne(x**2 + 2*x + 1, 0)

  >>> sp.solve(sp.simplify(sp.Implies(sp.Eq(x**2 + 2*x + 1, 0), sp.Eq(x, -1))))

  Traceback (most recent call last):
  File "", line 1, in 
  File "/home/user/.local/lib/python3.5/site-packages/sympy/solvers/solvers.py", line 1065, in solve
    solution = _solve(f[0], *symbols, **flags)
  File "/home/user/.local/lib/python3.5/site-packages/sympy/solvers/solvers.py", line 1401, in _solve
    f_num, sol = solve_linear(f, symbols=symbols)
  File "/home/user/.local/lib/python3.5/site-packages/sympy/solvers/solvers.py", line 1971, in solve_linear
    eq = lhs - rhs
  TypeError: unsupported operand type(s) for -: 'Or' and 'int'

  >>> sp.solveset(sp.simplify(sp.Implies(sp.Eq(x**2 + 2*x + 1, 0), sp.Eq(x, -1))))

  Traceback (most recent call last):
  File "", line 1, in 
  File "/home/user/.local/lib/python3.5/site-packages/sympy/solvers/solveset.py", line 880, in solveset
    raise ValueError("%s is not a valid SymPy expression" % (f))
  ValueError: Eq(x, -1) | Ne(x**2 + 2*x + 1, 0) is not a valid SymPy expression
none of the obvious ways appear to work. Does Sympy not support this kind of equational reasoning?

Re: Matrix Calculus

#24

Post by cjbillington »

The example has sine of a vector in it. I haven't had coffee yet today but I've never heard of being able to compute sine of a vector. What does this mean? Defining it by its Taylor series doesn't work like it does for square matrices because vectors can't be multiplied, and if you assume the direct product is what's meant then each term in the Taylor series is a different sized matrix and can't be added. Surely it doesn't mean elementwise sine?

Re: Matrix Calculus

#25

Post by svantana »

cjbillington wrote: post #24

The example has sine of a vector in it. I haven't had coffee yet today but I've never heard of being able to compute sine of a vector. What does this mean? Defining it by its Taylor series doesn't work like it does for square matrices because vectors can't be multiplied, and if you assume the direct product is what's meant then each term in the Taylor series is a different sized matrix and can't be added. Surely it d…

Yes it's element-wise, it's defined in the sidebar tab "Operators". In general the syntax seems closely aligned with Matlab, where all the math functions are elementwise.

Re: Matrix Calculus

#26

Post by leecarraher »

cjbillington wrote: post #24

The example has sine of a vector in it. I haven't had coffee yet today but I've never heard of being able to compute sine of a vector. What does this mean? Defining it by its Taylor series doesn't work like it does for square matrices because vectors can't be multiplied, and if you assume the direct product is what's meant then each term in the Taylor series is a different sized matrix and can't be added. Surely it d…

It is element-wise sin, which is not entirely correct. You can take the sin of, or any analytic function of a matrix, however you'd have to compute the characteristic polynomial for the matrix and apply the function to that. It is a result of the Cayley-Hamilton Theorem. here is a tutorial: http://web.mit.edu/2.151/www/Handouts/CayleyHamilton.pdf

Re: Matrix Calculus

#27

Post by btilly »

leecarraher wrote: post #26
cjbillington wrote: post #24

The example has sine of a vector in it. I haven't had coffee yet today but I've never heard of being able to compute sine of a vector. What does this mean? Defining it by its Taylor series doesn't work like it does for square matrices because vectors can't be multiplied, and if you assume the direct product is what's meant then each term in the Taylor series is a different sized matrix and can't be added. Surely it d…

It is element-wise sin, which is not entirely correct. You can take the sin of, or any analytic function of a matrix, however you'd have to compute the characteristic polynomial for the matrix and apply the function to that. It is a result of the Cayley-Hamilton Theorem. here is a tutorial: http://web.mit.edu/2.151/www/Handouts/CayleyHamilton.pdf

You don't need to know the characteristic polynomial to do it. Just sum as many terms of the power series as you want.

What the characteristic polynomial lets you do is to calculate it faster and more accurately because very large powers can be rewritten in terms of smaller powers that you already computed.

Re: Matrix Calculus

#29

Post by jey »

gajomi wrote: post #6
j7ake wrote: post #2

I never learned matrix calculus so I find this tool helpful for following technical papers that involve some matrix calculus

I have done all kinds of work that required some kind of matrix calculus in one form or another. There are of course all kinds of references (sibling links to my favorite), but I have found that more often than not really the best way to get the results you want is just to calculate them yourself. The work involved is usually tedious but trivial. But working through it goes along way to help make some sense of the va…

When I was starting out in machine learning, as a programmer with the most rudimentary calculus background, it was easy to derive algorithms that had terms like "gradient w.r.t X of log(det(inv(λI + A X A')))" which absolutely stumped me when trying to derive the gradient by hand by elementwise partials.

However, thanks to Minka's notes and the Matrix Cookbook, I was able to eventually get a handle on easy techniques for these derivations! It's certainly no substitute for getting a handle on the theory first by studying a textbook, but these pattern-matching shorthands are important practical techniques.

Re: Matrix Calculus

#30

Post by thearn4 »

Awesome, pretty handy to have for engineering/scientific computing code. Over time, you do start to build the intuition just like you do for scalar derivative calculations, but it takes time.
Post reply on HN