Pretty cool project!, I have been also trying to do something similar with very limited (abstract) OPs akin to fundamental computer instructions. Just using the numpy backend for now to test theory, but neat thing is that most of complexity lies in the abstract space like deciding which memory accesses could be coalesced even before generating the final code for a specific backend! As far as i know most of DL compilers struggle to generate optimum code, as model starts getting bigger and bigger . Halide project was/is a very cool project that speed up many kernels just by finding better cache/memory access pattern. If you happen to share more insights about your projects through blog-posts or whitepaper that would be really helpful.
Show HN: Luminal – Open-source, search-based GPU compiler
61–70 of 70 posts
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#62Earlier quoted context omitted.
Your description is exactly right. We create a search space of all possible kernels and find the best ones based on runtime. The best heuristic is no heuristic. This obviously creates a combinatorial problem that we mitigate with smarter search. The kernels are run on the computer the compiler is running on. Since runtime is our gold standard it will search for the best configuration for your hardware target. As long…
> that we mitigate with smarter search aka "a heuristic"
Exactly, I was going to ask about this bit…
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#63Re: Show HN: Luminal – Open-source, search-based GPU compiler
#64Earlier quoted context omitted.
Your description is exactly right. We create a search space of all possible kernels and find the best ones based on runtime. The best heuristic is no heuristic. This obviously creates a combinatorial problem that we mitigate with smarter search. The kernels are run on the computer the compiler is running on. Since runtime is our gold standard it will search for the best configuration for your hardware target. As long…
> that we mitigate with smarter search aka "a heuristic"
See my other comments about static profiling of kernels. There are ways of improving the search that keep runtime at the heart of it.
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#65Earlier quoted context omitted.
Your description is exactly right. We create a search space of all possible kernels and find the best ones based on runtime. The best heuristic is no heuristic. This obviously creates a combinatorial problem that we mitigate with smarter search. The kernels are run on the computer the compiler is running on. Since runtime is our gold standard it will search for the best configuration for your hardware target. As long…
> that we mitigate with smarter search aka "a heuristic"
mcts / rl isn't really a heuristic. but yes heuristics can be used temporarily to keep the search space small, and removed over time as the search algorithm improves.
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#66Earlier quoted context omitted.
very similar to superoptimisation, but most superoptimisers try to tackle turing-complete code. by just doing a very limited space of computation (linear algebra with 12 primitive ops) the search remains tractable. the search space is designed to remain logically equivalent at all times, by virtue of how its built (applying rewrite rules we know dont change the logical equivalence).
If the search space never leaves the programs that are equivalent to the original specification, that will probably limit the optimisations you can discover. (E.g. if you start out with standard matmul, you will not discover Strassen's algorithm.) This is not a criticism, I'm just trying to understand your algorithm.
could be...im not opposed to looking into this to see if there's no possible trajectory from naive to strassen's without leaving logical equivalency.
all the optimizations for matmul so far have been straightforward trajectories from naive (tiling, smem caching, tensor core offload, etc.)
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#67Earlier quoted context omitted.
If the search space never leaves the programs that are equivalent to the original specification, that will probably limit the optimisations you can discover. (E.g. if you start out with standard matmul, you will not discover Strassen's algorithm.) This is not a criticism, I'm just trying to understand your algorithm.
could be...im not opposed to looking into this to see if there's no possible trajectory from naive to strassen's without leaving logical equivalency. all the optimizations for matmul so far have been straightforward trajectories from naive (tiling, smem caching, tensor core offload, etc.)
There is an old CACM post that explains how to use a bit of randomness to avoid only doing semantics preserving program changes.
https://cacm.acm.org/research/stochastic-program-optimizatio...
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#68This might make a reasonably good correctness fuzzer for the underlying compiler. Lots of input code meant to calculate the same thing, report when a pair are found that calculate a different result.
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#69Earlier quoted context omitted.
Your description is exactly right. We create a search space of all possible kernels and find the best ones based on runtime. The best heuristic is no heuristic. This obviously creates a combinatorial problem that we mitigate with smarter search. The kernels are run on the computer the compiler is running on. Since runtime is our gold standard it will search for the best configuration for your hardware target. As long…
How long does this typically take? It sounds time consuming. Also, it seems like this could be similar to doing a GA?
[dead]
Re: Show HN: Luminal – Open-source, search-based GPU compiler
#70[dead]