Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tensor Comprehensions

research.fb.com

Post reply on HN
Print view View original

21–30 of 64 posts

Re: Tensor Comprehensions

#21

Post by amelius »

I think tensors are an overly crude way to model things. It's like computer science went back to the 60s and replaced all data structures by homogeneous blocks of memory.

Edit: of course a computer works best with blocks of memory; that doesn't mean a human developer should have the same view. As a simple example, think of the output vector of a classifier. Why is it a vector, and not a structure? Or think of the internals of an LSTM network; there is more structure in there than just tensors.

Re: Tensor Comprehensions

#22

Post by phaedrus »

This web page is also the first I've heard of Halide and Polyhedral Compilation. This is exciting to me because I've been working on relational (database) data and logic comprehensions, and in a case of convergent evolution Halide looks a lot like my notation and Polyhedral Compilation looks much like diagrams I've been drawing on my whiteboard. Where can I learn more on this?

Re: Tensor Comprehensions

#23

Post by ozinenko »

phaedrus wrote: post #22

This web page is also the first I've heard of Halide and Polyhedral Compilation. This is exciting to me because I've been working on relational (database) data and logic comprehensions, and in a case of convergent evolution Halide looks a lot like my notation and Polyhedral Compilation looks much like diagrams I've been drawing on my whiteboard. Where can I learn more on this?

The paper has a couple of useful references.

Otherwise, we have a site with general information on polyhedral compilation http://polyhedral.info/ and Halide has its own site http://halide-lang.org/

Re: Tensor Comprehensions

#25

Post by PeterisP »

amelius wrote: post #21

I think tensors are an overly crude way to model things. It's like computer science went back to the 60s and replaced all data structures by homogeneous blocks of memory. Edit: of course a computer works best with blocks of memory; that doesn't mean a human developer should have the same view. As a simple example, think of the output vector of a classifier. Why is it a vector, and not a structure? Or think of the int…

Tensors are the way to effectively execute things - modeling things is about the "human interface" to the data, but when you need to do stuff with very, very, very large quantities of it at a good performance, then you'd want the system to transform the data and desired operations from your "human-friendly" model to a "machine-friendly" model, and that's generally going to be homogenous blocks of memory, and "vectorizing" processing as much as possible so you have less or no item-specific logic but instead have matrix operations that do the same thing to many data items at once in parallel.

It's just like with OOP in game programming where performance matters - even if you want a nice object model for programmer convenience, you'd also want to ensure that you can store the object data into a homogeneous array sequentially in memory, as that gets you a major performance impact; I seem to recall that Carmack had an in-detail article about that some time ago, but can't easily find it.

Re: Tensor Comprehensions

#26

Post by bllguo »

wrs wrote: post #2

>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…

I've seen and heard 'codes' in scientific computing, but nowhere else. Seems to support your hypothesis

Re: Tensor Comprehensions

#27

Post by grondilu »

From the documentation on arxiv:

> Variables not defined anywhere, implicitly become index variables.

That seems like a bold choice. Wasn't there a trend in programming languages, even very high level ones, to encourage variable declaration?

Re: Tensor Comprehensions

#28

Post by liuliu »

smhx wrote: post #5
falcor84 wrote: post #3

Could someone please explain how this compares to the TensorFlow approach? I can only assume that it's omitted from the article due to marketing reasons.

Tensor Comprehensions are not a deep learning framework, but a way to write new optimized operators for deep learning. Hence, Tensor Comprehensions can be integrated with TensorFlow.

There is XLA though: https://www.tensorflow.org/performance/xla/

But I guess unlike TC / Halide / TVM, that is less generic and much less customization.

Re: Tensor Comprehensions

#29

Post by username223 »

fdej wrote: post #17
wrs wrote: post #2

>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…

Yes, this is just jargon specific to the HPC / numerical analysis field.

And not without meaning. HPC people hold themselves to higher standards than the average Joes throwing stuff at the currently-fashionable "continuous integration server" until the lights turn green. "Codes" are often things that have been proven, written on a legal pad, typed into the machine, and finally validated on a set of test problems. Could you prove that your program has second-order numerical stability? https://en.wikipedia.org/wiki/Numerical_stability

Re: Tensor Comprehensions

#30

Post by ribs »

wrs wrote: post #2

>produce the high-performance codes that the machine learning community needs Somewhat OT, but I've been wondering for a long time… Is the HPC community the only place the word "codes" is used like this? In usual CS parlance programming is done using a substance called "code" ("the high-performance code the community needs"), but in HPC literature the word "codes" is used, as if programming consisted of distinct obje…

I started working with HPC users three years ago and yes, they're the only ones who used "codes" that way.

At least, the only ones in the last 30 years, I'm figuring - there is a reason we call it "code", and it's an old one!

Post reply on HN