Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Tensor Algebra Compiler

tensor-compiler.org

Post reply on HN
Print view View original

31–37 of 37 posts

Re: The Tensor Algebra Compiler

#31

Post by fredrikbk »

110011 wrote: post #15
fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

I entered "x(i) = y(i) + z(i)" in the online demo, all three sparse, and I noticed some useless variable definitions: int32_t iz0 = z1_idx[pz1]; Is this intentional? (if so, why)

We’ve tried to engineer it to emit clean code, but seems we left in some unused variables.

The C compilers dead code elimination should remove those though.

Re: The Tensor Algebra Compiler

#32

Post by quadruplebond »

philipkglass wrote: post #5
fredrikbk wrote: post #2

Hi Hacker News! I’m one of the developers. This project was also featured in MIT News yesterday: http://news.mit.edu/2017/faster-big-data-analysis-tensor-alg... The code is available at: https://github.com/tensor-compiler/taco/issues I’m happy to discuss the project and to answer any questions :)

The MIT story focuses on "big data," but it looks like this might be applicable to coupled-cluster calculations in physics/chemistry too. Is it? Can you compare/contrast with e.g. the Cyclops Tensor Framework ( http://solomon2.web.engr.illinois.edu/ctf/ ) or NWChem's Tensor Contraction Engine ( http://www.csc.lsu.edu/~gb/TCE/ )?

This looks like it might be nice, but regular coupled cluster doesn't really have very many sparse matrices so mapping things into Blas3 is fine. Also if you go the reduced scaling route and use PNOs or OSVs everything is mapped into matrix matrix for CCSD anyways.

Re: The Tensor Algebra Compiler

#33

Post by fredrikbk »

gugagore wrote: post #29

Conventional wisdom has that dense matrix-matrix multiply is generally faster if you use some cache-aware scheme instead of the textbook 3 nested for loops. Is there a similar story for sparse matrices ever?

The story for sparse matrices is more complicated. Because of the dependencies imposes by sparse data structures (you don’t have random access and often have to traverse them) you cannot do loop ruling without adding if statements (I think you can probably get rid of these by preconputing, e.g. row halfway points). It may still make sense, but there’s a bigger cost. However, you can lay out data in a tiled way to get better cache blocking; taco lets you do this by storing a blocked matrix as a 4-tensor.

Because of the lack of random access, some algorithm like linear combination matrix-matrix multiplication benefits from adding a dense workspace that gives you a view into one row. Then you can scatter into it and when you’re done copy the nonzeroes to the sparse result matrix. This algorithm is sometimes called Gustavson’s algorithm after the person who published it first). We have worked out this optimization within the taco framework, it applies to other kernels too, and are now writing it up for publication.

Re: The Tensor Algebra Compiler

#36

Post by throwaway613834 »

surban wrote: post #24
throwaway613834 wrote: post #21

Earlier quoted context omitted.

I'm not sure if this is a sensible question, but what is the use of this compared to (say) an autodiff library like FABAD++ [1]? Is performance the main advantage, or expressive power, or something else? [1] http://www.fadbad.com/fadbad.html

If you use autodiff and don't have an 1:1 relationship between function and argument indices you might have to do atomic sums or locking when computing the derivative because multiple elements of the function derivative correspond to one element of the argument derivative. On a CPU this might be okay but on CUDA GPUs this usually has a performance impact. Thus we transform the derivatives so that we have an explicit…

Oh wow, interesting. Thanks!
Post reply on HN