Halide, a language for image processing and computational photography

The end result is not like OpenCL, CUDA/PTX, or RenderScript. (All of which we regard as suitable target languages for Halide. PTX (CUDA's low-level assembly language) and OpenCL are currently supported.) As a concrete example, there is no explicit memory allocation in Halide and loops are often implicit. The overall character of Halide is that of a functional programming language.

As another example, segmentation violations due to errors in Halide code are impossible, barring bugs in the compiler implementation or errors in user provided C++ code, which Halide has no control over. Some set of bounds errors are reported at compile time and the rest turn into (efficient) assertion checks at runtime.

Schedules do get pretty low-level and writing them is difficult. Algorithms are generally a fair bit easier to write and while there is plenty to be done in improving the language, it is already a step up in productivity over e.g. OpenCL or even straight C++ for a lot of tasks within the domain of data parallel processing.

I gave up relying on magic compilers a long time ago. And having worked in this domain for a long time I'm actually offended by people who write off the problem as simply a matter of a good enough optimizer. This has significantly held back both performance and portability in imaging. (And likely other areas.)

Halide is not magic, it is just a better slicing of the problem backed up by a good implementation. As always there is no free lunch but when it comes down to actually shipping this kind of code across a wide variety of platforms with great performance, it is a lot more productive than anything else out there.

The target image processing algorithms are low level. Think "How can I write the fastest 3x3 box blur for Sandy Bridge and later architectures", not "How can I speed up my face detector".

Examples of scheduling decisions that Halide deals with: process image in 4x4? 8x8? 16x16 chunks? use a temporary buffer for an intermediate processing stage, causing worse cache behavior but reusing results that are needed more than once? use a sliding window the size of a couple rows as a compromise?

This kind of work means the difference between a laggy or interactive Gaussian Blur radius slider in Photoshop.

The Halide code shown at the talk was replacing really hard-to-read code with lots of SIMD compiler intrinsics. Dozens of lines of code doing something that would be 5 lines in a naive implementation. With Halide, it's almost as readable as the naive version because the scheduling stuff is separated from the algorithm.

For an application like Photoshop, this is a big win because they will never choose code readability over performance. Performance wins every time. If they can get the same performance with readable code, they are very happy.

GPU code generation falls naturally out of the scheduling intelligence and restricted problem domain.

I have never used Halide, so I do not intend to endorse it, but this line of inquiry is absolutely useful for a certain niche of programming.