Live data from Hacker News

Tensor Comprehensions

research.fb.com

51–60 of 64 posts

Re: Tensor Comprehensions

#51
post #50
post #35

Earlier quoted context omitted.

In the Commonwealth we say maths but we still say code. I've noticed we also tend to also write "computer program" the US way, despite writing "TV programme".

Yup, British-educated here: programme for sequences of activities, program for the thing the computer runs... it’s just a little oddity I’ve noticed I’ve picked up and seem to be quite consistently applying.

Yep. Even though a computer program is... a sequence of activities.

Re: Tensor Comprehensions

#52
post #50
post #35

Earlier quoted context omitted.

In the Commonwealth we say maths but we still say code. I've noticed we also tend to also write "computer program" the US way, despite writing "TV programme".

Yup, British-educated here: programme for sequences of activities, program for the thing the computer runs... it’s just a little oddity I’ve noticed I’ve picked up and seem to be quite consistently applying.

Yes... and I use ”to programme” to denote the activity of planning activities, and ”to program” to indicate the process of coding instructions into a computer. I would probably use ”programme to program” if I ever had to discuss the idea of planning one’s intentions to issue instructions to a machine.

Re: Tensor Comprehensions

#53
post #42

Earlier quoted context omitted.

And not without meaning. HPC people hold themselves to higher standards than the average Joes throwing stuff at the currently-fashionable "continuous integration server" until the lights turn green. "Codes" are often things that have been proven, written on a legal pad, typed into the machine, and finally validated on a set of test problems. Could you prove that your program has second-order numerical stability? http…

A shining example of the Dunning–Kruger effect. Scientific code is some of the worst-engineered code you'll ever deal with because it's written by people that are great at physics but terrible at software engineering. It will be poorly documented, have single letter var names everywhere without comments, have a bunch of hidden dependencies on whatever the postdoc had installed on his/her laptop, leak memory but 'term…

Sure, the code is usually pretty ugly, but it also tends to work. The people who write it view software as just another tool, so they care more about working than looking pretty.

> Oh, and there will almost never be even a single regression test.

Bullshit. "Codes" tend to be run against analytical and/or experimental results.

Re: Tensor Comprehensions

#54

This web page is also the first I've heard of Halide and Polyhedral Compilation. This is exciting to me because I've been working on relational (database) data and logic comprehensions, and in a case of convergent evolution Halide looks a lot like my notation and Polyhedral Compilation looks much like diagrams I've been drawing on my whiteboard. Where can I learn more on this?

The TC paper uses techniques that are in the R-Stream compiler. You can get more information about the R-Stream compiler at https://www.reservoir.com/research/publications/?cat=35 - the first paper there also validates the use of polyhedral compilation for deep learning last year. I particularly recommend the article about R-Stream in the Encyclopedia of Parallel Compilation, David Padua (ed). https://www.reservoir.com/publication/r-stream-compiler-2/

Re: Tensor Comprehensions

#55

This web page is also the first I've heard of Halide and Polyhedral Compilation. This is exciting to me because I've been working on relational (database) data and logic comprehensions, and in a case of convergent evolution Halide looks a lot like my notation and Polyhedral Compilation looks much like diagrams I've been drawing on my whiteboard. Where can I learn more on this?

The R-Stream Encyclopedia article talks about how to raise to the polyhedral model from SSA; this is no problem especially for the super regular computations in Tensor computations used in deep learning. If you get in touch with me I can also get you a copy of the 2008 paper about R-Stream that described it: https://www.reservoir.com/publication/final-report-r-stream-...

By raising from C you don't need to use or learn a new language. Or one can generate C from a succinct notation or framework, and polyhedral optimize from there. That is what was in the R-Stream-TF paper.

Re: Tensor Comprehensions

#56

This web page is also the first I've heard of Halide and Polyhedral Compilation. This is exciting to me because I've been working on relational (database) data and logic comprehensions, and in a case of convergent evolution Halide looks a lot like my notation and Polyhedral Compilation looks much like diagrams I've been drawing on my whiteboard. Where can I learn more on this?

Also, we're hiring: https://www.reservoir.com/company/careers/

Re: Tensor Comprehensions

#58
I could not find how to integrate this into a C++ program. Does it need ATen or is there a lower level of integration? Is there even the possibility of getting C bindings?

Re: Tensor Comprehensions

#59
post #58

I could not find how to integrate this into a C++ program. Does it need ATen or is there a lower level of integration? Is there even the possibility of getting C bindings?

Tensor Comprehensions does not try to own memory allocation and CPU/GPU transfers. ATen is one simple way of getting that, which we used for tests. Anything convertible to DLPack tensors should work as long as nothing fancy happens with tensor shapes.

C bindings don't seem to be a priority.

Feel free to use the contacts provided in the documentation here: https://facebookresearch.github.io/TensorComprehensions/cont....

Re: Tensor Comprehensions

#60

Out of curiosity, is there any overlap with objectives or performance of accelerate https://github.com/AccelerateHS/accelerate ?

Tensor Comprehensions is mostly targeted at arithmetics operations that appear in DL workloads, and the notation strives to be usable for DL experts. Polyhedral optimizer is oriented towards imperative languages, so we won't end up doing the same optimizations. Really hard to compare. The spirit of making parallel programming simpler is common :)
Post reply on HN