Tensor Comprehensions
research.fb.com
Tensor Comprehensions
1–10 of 64 posts
Re: Tensor Comprehensions
#2>produce the high-performance codes that the machine learning community needs
Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct objects. Does this arise from some divergent history (would I have called my LINPACK library punched card deck a "code"?) or what?
Re: Tensor Comprehensions
#3Could someone please explain how this compares to the TensorFlow approach? I can only assume that it's omitted from the article due to marketing reasons.
Re: Tensor Comprehensions
#4>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…
Code is generally considered a mass noun among software engineers, but "codes" is pretty commonly used by academics, especially in other disciplines. In particular, physicists and mathematicians seem to use it pretty frequently, so that might explain why some in the HPC community use it as well.
I've also noticed that it seems more common among Europeans, but that might be just personal experience.
Re: Tensor Comprehensions
#5Could someone please explain how this compares to the TensorFlow approach? I can only assume that it's omitted from the article due to marketing reasons.
Tensor Comprehensions are not a deep learning framework, but a way to write new optimized operators for deep learning. Hence, Tensor Comprehensions can be integrated with TensorFlow.
Re: Tensor Comprehensions
#6Could someone please explain how this compares to the TensorFlow approach? I can only assume that it's omitted from the article due to marketing reasons.
My understanding is that Tensor Comprehensions provides a way to automatically generate optimized CUDA code for algorithms written in a high-level language that more closely mirrors the notation used in mathematical formulas. So you could use it to automatically find more optimal low-level implementations for components used in libraries such as PyTorch and TensorFlow which usually call out to hand-written low-level implementations.
Re: Tensor Comprehensions
#7I'm a fan of evolutionary algorithms, but are they really effective enough here to be comparable to an engineer tuning code? They might be able to find a good configuration of a few canned options but real optimization often requires some creativity or at least an understanding of the hardware. Will certainly be interesting to see this in practice!
Re: Tensor Comprehensions
#8>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…
Whenever I read or that, I assume it was a language issue. As a native English speaker "codes" sounds like a mistake.
Re: Tensor Comprehensions
#9Slightly resembles the Tensor Contraction Engine for quantum chemistry/physics (http://www.csc.lsu.edu/%7Egb/TCE/ ). Although it predates this by a couple of decades.
Re: Tensor Comprehensions
#10I'm a fan of evolutionary algorithms, but are they really effective enough here to be comparable to an engineer tuning code? They might be able to find a good configuration of a few canned options but real optimization often requires some creativity or at least an understanding of the hardware. Will certainly be interesting to see this in practice!
[deleted]