Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What I wish someone had told me about tensor computation libraries

eigenfoo.xyz

Post reply on HN
Print view View original

31–40 of 89 posts

Re: What I wish someone had told me about tensor computation libraries

#31

Post by yudlejoza »

37ef_ced3 wrote: post #7

NN-512 ( https://NN-512.com ) Generate fully vectorized, stand-alone, human-readable C99 code for neural net inference, and understand exactly what's happening. For example, watch the code run with Linux's perf top and see the relative costs of each layer of the computation. Total transparency, no dependencies outside the C POSIX library

Great. Thanks!

1. Any particular reason you chose to avoid GPUs?

2. Did you benchmark your code's performance against GPU-centric codes (ideally for the same problem and problem-size)?

Re: What I wish someone had told me about tensor computation libraries

#32

Post by godelski »

dragandj wrote: post #23
godelski wrote: post #22

Earlier quoted context omitted.

I'd actually love that material in C++/CUDA.

If only C++ supported interactive REPL and the rest of Clojure/Lisp goodies, that might be possible. However, the code is CLOSELY related to the actual CUDA/C++ api. It's a lot simpler, concise, and everything, but I explain everything so that you can use the relevant parts with cuDNN and DNNL APIs in any language that you're most proficient in.

Thanks! That's helpful to know. I have no experience with Clojure/Lisp but a fair amount with C/C++ (minor in CUDA).

Re: What I wish someone had told me about tensor computation libraries

#33

Post by 37ef_ced3 »

yudlejoza wrote: post #31
37ef_ced3 wrote: post #7

NN-512 ( https://NN-512.com ) Generate fully vectorized, stand-alone, human-readable C99 code for neural net inference, and understand exactly what's happening. For example, watch the code run with Linux's perf top and see the relative costs of each layer of the computation. Total transparency, no dependencies outside the C POSIX library

Great. Thanks! 1. Any particular reason you chose to avoid GPUs? 2. Did you benchmark your code's performance against GPU-centric codes (ideally for the same problem and problem-size)?

The goal of NN-512 is efficient neural net inference on inexpensive, CPU-only cloud compute instances

For example, a Skylake-X cloud compute instance costs $10 per CPU-core per month at Vultr, and the NN-512 generated code does about 18 DenseNet121 inferences per CPU-core per second (in series, not batched)

In contrast, GPU cloud compute is almost unbelievably expensive. Even Linode charges $1000 per month, or $1.50 per hour (look at the GPU plans: https://www.linode.com/pricing/#row--compute)

As AVX-512 becomes better supported by Intel and AMD chips, it becomes more attractive as an alternative to expensive GPU instances for workloads with small amounts of inference mixed with other computation

Re: What I wish someone had told me about tensor computation libraries

#34

Post by prideout »

Are these libraries ever useful in non-deep learning applications? It sounds like Theano is a bit more general purpose, but why would I ever need it outside of a deep learning context?

I wonder if it could be used for something crazy, e.g. setting up a graph that generates shadertoy-like images on the GPU.

Re: What I wish someone had told me about tensor computation libraries

#35

Post by 6gvONxR4sf7o »

prideout wrote: post #34

Are these libraries ever useful in non-deep learning applications? It sounds like Theano is a bit more general purpose, but why would I ever need it outside of a deep learning context? I wonder if it could be used for something crazy, e.g. setting up a graph that generates shadertoy-like images on the GPU.

They are. Lots of numerical code benefits from GPU and lots of numerical code benefits from derivatives. Simulations, solvers, numerical optimization, good old fashioned statistics.

Re: What I wish someone had told me about tensor computation libraries

#36

Post by yudlejoza »

37ef_ced3 wrote: post #33
yudlejoza wrote: post #31

Earlier quoted context omitted.

Great. Thanks! 1. Any particular reason you chose to avoid GPUs? 2. Did you benchmark your code's performance against GPU-centric codes (ideally for the same problem and problem-size)?

The goal of NN-512 is efficient neural net inference on inexpensive, CPU-only cloud compute instances For example, a Skylake-X cloud compute instance costs $10 per CPU-core per month at Vultr, and the NN-512 generated code does about 18 DenseNet121 inferences per CPU-core per second (in series, not batched) In contrast, GPU cloud compute is almost unbelievably expensive. Even Linode charges $1000 per month, or $1.50…

I'm not disagreeing with you. I acknowledge that there may be a market for CPU-only NN tasks.

I think a thorough benchmark, either by you or by someone else, will only help your case, by giving a clear picture to those who need to make a decision.

Fun fact, GPUs are massively under-utilized during NN training. So it's quite possible NN on a good CPU might be only slightly slower.

Re: What I wish someone had told me about tensor computation libraries

#37

Post by ssivark »

yudlejoza wrote: post #31
37ef_ced3 wrote: post #7

NN-512 ( https://NN-512.com ) Generate fully vectorized, stand-alone, human-readable C99 code for neural net inference, and understand exactly what's happening. For example, watch the code run with Linux's perf top and see the relative costs of each layer of the computation. Total transparency, no dependencies outside the C POSIX library

Great. Thanks! 1. Any particular reason you chose to avoid GPUs? 2. Did you benchmark your code's performance against GPU-centric codes (ideally for the same problem and problem-size)?

GPUs are typically useful for training (due to massive parallelism), but not for inference.

Re: What I wish someone had told me about tensor computation libraries

#38

Post by dragandj »

godelski wrote: post #32
dragandj wrote: post #23

Earlier quoted context omitted.

If only C++ supported interactive REPL and the rest of Clojure/Lisp goodies, that might be possible. However, the code is CLOSELY related to the actual CUDA/C++ api. It's a lot simpler, concise, and everything, but I explain everything so that you can use the relevant parts with cuDNN and DNNL APIs in any language that you're most proficient in.

Thanks! That's helpful to know. I have no experience with Clojure/Lisp but a fair amount with C/C++ (minor in CUDA).

Please read a few of tutorials from my blog. Most programmers in your situation told me that they had no problems following it; it only gradually introduces advanced Clojure concepts, and code snippets are usually extremely short + completely executable interactively as-is.

Re: What I wish someone had told me about tensor computation libraries

#39

Post by hansvm »

wokwokwok wrote: post #29
dragandj wrote: post #14

Let me chip in with some self-promotion. This book explains and executes every single line of code interactively, from low level operations to high-level networks that do everything automatically. The code is built on the state of the art performance operations of oneDNN (Intel, CPU) and cuDNN (CUDA, GPU). Very concise readable and understandable by humans. https://aiprobook.com/deep-learning-for-programmers/ Here's…

Concise isn’t always better. You’re throwing alway all the names of the arguments and using arbitrary words like “conv” to represent operations. This is typical bad clojure in my experience; write once, forget wtf the magic was, throw away and rewrite it again later. Clojure doesn’t have to be incomprehensible arcane magic that does everything in 10 lines. The more complex the code, the more important it is that what…

Concision is a style choice to be used with care. Spending screen space on additional characters and descriptions detracts from the ability to fit more logic on the screen at once and grok the larger flow. Splashing symbolic alphabet soup into your IDE in the name of concision isn't usually a good idea, but naming something "conv" in the immediate local context of a convolutional layer doesn't seem so bad.
Post reply on HN