Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tensor Comprehensions

research.fb.com

Post reply on HN
Print view View original

31–40 of 64 posts

Re: Tensor Comprehensions

#31

Post by nicovasilache »

phaedrus wrote: post #22

This web page is also the first I've heard of Halide and Polyhedral Compilation. This is exciting to me because I've been working on relational (database) data and logic comprehensions, and in a case of convergent evolution Halide looks a lot like my notation and Polyhedral Compilation looks much like diagrams I've been drawing on my whiteboard. Where can I learn more on this?

@phaedrus take a look at Weld from Stanford then, may also be useful to you. Edit: in addition to what @ozinenko pointed to of course..

Re: Tensor Comprehensions

#32

Post by nicovasilache »

grondilu wrote: post #27

From the documentation on arxiv: > Variables not defined anywhere, implicitly become index variables. That seems like a bold choice. Wasn't there a trend in programming languages, even very high level ones, to encourage variable declaration?

This is one reason I also prefer to call TC a notation personally. We can't allocate and declare inside TC, that may change in the future but for now we went for the easiest entry point to Halide and polyhedral IR we could think of. You can lower simple C loops or other real languages in those IRs too but the programs are so much terser in TC that we have come to expect terseness.

Re: Tensor Comprehensions

#33

Post by nicovasilache »

tomrod wrote: post #24

Neat! This needs Python bindings, stat!

Python bindings are in there :) This needs a tensor library callable from python that works on GPUs. One direction we are going towards is PyTorch via ATen / Torch tensors; we already use the C++ parts of ATen. Of course any other CUDA tensor library with minimal alloc/copy/synchronize would work too. Send a PR? ;)

Re: Tensor Comprehensions

#34

Post by [deleted] »

nicovasilache wrote: post #33
tomrod wrote: post #24

Neat! This needs Python bindings, stat!

Python bindings are in there :) This needs a tensor library callable from python that works on GPUs. One direction we are going towards is PyTorch via ATen / Torch tensors; we already use the C++ parts of ATen. Of course any other CUDA tensor library with minimal alloc/copy/synchronize would work too. Send a PR? ;)

[deleted]

Re: Tensor Comprehensions

#35

Post by Nition »

r00fus wrote: post #16
grandmczeb wrote: post #4

Earlier quoted context omitted.

Code is generally considered a mass noun among software engineers, but "codes" is pretty commonly used by academics, especially in other disciplines. In particular, physicists and mathematicians seem to use it pretty frequently, so that might explain why some in the HPC community use it as well. I've also noticed that it seems more common among Europeans, but that might be just personal experience.

code:codes::math:maths?

In the Commonwealth we say maths but we still say code.

I've noticed we also tend to also write "computer program" the US way, despite writing "TV programme".

Re: Tensor Comprehensions

#36

Post by [deleted] »

nicovasilache wrote: post #33
tomrod wrote: post #24

Neat! This needs Python bindings, stat!

Python bindings are in there :) This needs a tensor library callable from python that works on GPUs. One direction we are going towards is PyTorch via ATen / Torch tensors; we already use the C++ parts of ATen. Of course any other CUDA tensor library with minimal alloc/copy/synchronize would work too. Send a PR? ;)

[deleted]

Re: Tensor Comprehensions

#37

Post by priy2201 »

tomrod wrote: post #24

Neat! This needs Python bindings, stat!

for clarity, we provide very "basic" python bindings. For example checkout https://facebookresearch.github.io/TensorComprehensions/mapp... for simple example of how we expose mapping options to python.

Also TC doesn't do data allocation itself and it requires tensor library from users to do that. So if you are using Caffe2 for example, you could use the TcOp that we ship inside TensorComprehensions for Caffe2 and using caffe2 pybindings, you can already write TC in python. No work needed for creating python bindings. We welcome PRs on this :)

As for other tensor libraries, like ATen based on TH* used in torch/pytorch, you have the ability to create tensor but these require integrating tensor library into TC and writing pybindings for them. PRs welcome on this as well.

Re: Tensor Comprehensions

#38

Post by stochastic_monk »

grandmczeb wrote: post #4
wrs wrote: post #2

>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…

Code is generally considered a mass noun among software engineers, but "codes" is pretty commonly used by academics, especially in other disciplines. In particular, physicists and mathematicians seem to use it pretty frequently, so that might explain why some in the HPC community use it as well. I've also noticed that it seems more common among Europeans, but that might be just personal experience.

"Codes" tends to refer to error-correcting codes or encoding formats in my experience, while code refers to source or assembly code for machine execution.

One is a concept/idea/format while the other is a set of instructions.

Re: Tensor Comprehensions

#39

Post by evincarofautumn »

HumanDrivenDev wrote: post #8
wrs wrote: post #2

>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…

Whenever I read or that, I assume it was a language issue. As a native English speaker "codes" sounds like a mistake.

It’s common in Indian English, as in “I have written a code to do the needful, but I have a doubt about it that perhaps you could answer.” Outside that dialect I’d call it an error.

Re: Tensor Comprehensions

#40

Post by ychen306 »

tehsauce wrote: post #7

I'm a fan of evolutionary algorithms, but are they really effective enough here to be comparable to an engineer tuning code? They might be able to find a good configuration of a few canned options but real optimization often requires some creativity or at least an understanding of the hardware. Will certainly be interesting to see this in practice!

Hand-tuning code doesn't scale to the kind of networks people write.
Post reply on HN