The Tensor Algebra Compiler • Hacker News Board

Re: The Tensor Algebra Compiler

#11

Post by nmca » Wed, Nov 01, 2017, 8:03 PM UTC

Hm, the lack of GPU-specific support seems unfortunate. Cool work though, the formulation as a lattice is pretty sweet :)

Re: The Tensor Algebra Compiler

#12

Post by 110011 » Wed, Nov 01, 2017, 8:04 PM UTC

fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

Just saw your talk. Man you are one hell of a speaker. Great slides, animations and presentation. I enjoyed it.

It seems like a strange thing that no one thought of doing this before! Nice that you identified a cool problem and solved it :)

Re: The Tensor Algebra Compiler

#13

Post by scottlegrand2 » Wed, Nov 01, 2017, 8:20 PM UTC

110011 wrote: post #12

fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

Just saw your talk. Man you are one hell of a speaker. Great slides, animations and presentation. I enjoyed it. It seems like a strange thing that no one thought of doing this before! Nice that you identified a cool problem and solved it :)

Not entirely true. Sparse x Dense Matrix is an important component of the DSSTNE deep learning framework:

https://github.com/amzn/amazon-dsstne

But this library is clearly more thorough.

Re: The Tensor Algebra Compiler

#14

Post by floatboth » Wed, Nov 01, 2017, 8:59 PM UTC

nmca wrote: post #11

Hm, the lack of GPU-specific support seems unfortunate. Cool work though, the formulation as a lattice is pretty sweet :)

Yeah, would be nice to see one of these compute frameworks compile to SPIR-V and run on Vulkan…

Re: The Tensor Algebra Compiler

#15

Post by 110011 » Wed, Nov 01, 2017, 9:09 PM UTC

fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

I entered "x(i) = y(i) + z(i)" in the online demo, all three sparse, and I noticed some useless variable definitions: int32_t iz0 = z1_idx[pz1];

Is this intentional? (if so, why)

Re: The Tensor Algebra Compiler

#16

Post by comnetxr » Wed, Nov 01, 2017, 9:52 PM UTC

Often (in the scientific computing communities that I am in) large tensor contractions are done by reshaping tensors into matrices, transposing, and using matrix multiplication. The contraction time can change dramatically based on the order one contracts the tensors and the relative sizes of the tensors. It seems this just uses a large nested for loop - how does this strategy compare to using dedicated matrix multiplication algorithms?

Re: The Tensor Algebra Compiler

#17

Post by mattnewton » Wed, Nov 01, 2017, 10:20 PM UTC

fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

Skimming it quickly I didn’t see anything about gpu or vector instruction support as compilation targets. Is this planned? Did I miss something and this is at a higher layer?

Ps: excellent name

Re: The Tensor Algebra Compiler

#18

Post by surban » Wed, Nov 01, 2017, 10:44 PM UTC

We did some work on computing derivative expressions (goal is application to deep learning) for such tensor algebras. I was going to release it on arXiv in the next few weeks, but now seems to be a good time. Here you go (preliminary version):

https://github.com/surban/TensorAlgDiff/raw/master/elemdiff....

Our system takes a tensor algebra (we call it element-wise defined tensor) and outputs expressions for the derivatives w.r.t. all of its arguments. The expression may contain sums and the indices of the argument tensors can be any linear combination of the function indices (see our example for more details). It correctly handles the cases where an index does not appear in an argument, appears twice, appears as (i+j) etc.

Code to play with at https://github.com/surban/TensorAlgDiff

Re: The Tensor Algebra Compiler

#19

Post by blurbleblurble » Wed, Nov 01, 2017, 10:47 PM UTC

Awesome!

I made something similar recently, a GLSL code printer for for SymPy. I use it for generating GLSL out of Clifford algebras. I'm using it to do higher dimensional & conformal geometry.

https://github.com/sympy/sympy/pull/12713

Re: The Tensor Algebra Compiler

#20

Post by amelius » Wed, Nov 01, 2017, 10:47 PM UTC

fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

Hi, I just saw the presentation too, and it looks impressive.

I'm hoping that you can (at some point) write a language-agnostic library, so that your work can be used in many other development tools (which are not using C++). I suppose it would amount to publishing the documentation of the intermediate code you are already using. This would also save you the trouble of writing and maintaining a GPU back-end because other people could do that using your library.

Anyway, great work!