Viewing profile — jafioti

i did a quick scroll and was happy to see a long-ish article on XLA and TPUs. then i realized it was literally just "using vmap for parallel loops is better than fori", but in mass…

That's a bit trite tbh. We all know of these techniques, but actually implementing them on GPUs in a low-overhead manner that maintains the model's fidelity is challenging. It's mu…

Thats reasonably accurate, we're fusing both pre-defined operations as well as codegenned operations. Block-level operations live inside the search space, as do kernel, warp and th…

could be...im not opposed to looking into this to see if there's no possible trajectory from naive to strassen's without leaving logical equivalency. all the optimizations for matm…

mcts / rl isn't really a heuristic. but yes heuristics can be used temporarily to keep the search space small, and removed over time as the search algorithm improves.

np-hard is still solveable with constraints. look at go.

you suppose correctly ;)

yep, parallelized profiling across many devices is definitely something i want to add.

hopefully! i dont know the exact trick they used, but the idea is to design the search space such that that trick is discoverable.

yup! we build a search space by iteratively applying rewrite rules in every possible order (using e-graphs to do this efficiently). the rewrites alter stuff like looping / tiling s…

e-graphs are awesome! none of this would be possible without them.

we're working on techniques like mcts and RL (e.g. AlphaGo) to manage the search space, but you'd be suprised how far you can get if you carefully design the search space to preven…

ah i see the confusion. we do common subexpression elimination of the terms in the search space (which allows single application of rewrites to apply to many repeat patterns) but t…

very similar to superoptimisation, but most superoptimisers try to tackle turing-complete code. by just doing a very limited space of computation (linear algebra with 12 primitive …

we're just optimizing linear algebra, which is mostly made up of patterns of simple ops. for instance, matmul is just broadcasted multiply -> sum reduce. the search does common sub…

basically autotuning on steroids. instead of searching single dimensions of optimization (tile sizing, etc.) we search through full algebraic rewrites (like rewriting softmax to on…

a lot of the ideas luminal is built on are here: https://arxiv.org/abs/2304.04332

yep! currently we're emitting cuda / metal but once the search is better, i want to directly emit ptx / low-level asm on other hardwares.

a lot of the search is still being optimized so we don't match super hand-optimized kernels like llama.cpp has, so we def don't match their tps yet, but i want to make a perf track…

we do support tensor cores, but the ops are only part of the search space, so there's virtually no overhead for them. the frontend and main ir is only 12 ops, and we can add hardwa…

Hi HN, I’m Joe. My friends Matthew, Jake and I are building Luminal ( https://luminalai.com/ ), a GPU compiler for automatically generating fast GPU kernels for AI models. It uses …