Outperforming LAPACK with C++ metaprogramming
wordsandbuttons.online
Outperforming LAPACK with C++ metaprogramming
1–10 of 82 posts
Re: Outperforming LAPACK with C++ metaprogramming
#2I wonder if the similar tricks work in other languages with similarily optimizing compilers, but "nicer" metaprogramming, such as Common LISP, Rust and perhaps OCaml? (with camlp4)
Even more interesting would be whether this leads to the same huge performance gains in those languages as well.
Re: Outperforming LAPACK with C++ metaprogramming
#3I wonder if the similar tricks work in other languages with similarily optimizing compilers, but "nicer" metaprogramming, such as Common LISP, Rust and perhaps OCaml? (with camlp4) Even more interesting would be whether this leads to the same huge performance gains in those languages as well.
Or even Dlang .
Re: Outperforming LAPACK with C++ metaprogramming
#4Please check with array initialized from file or other dynamic source. With maximal inlining, there's a risk of constant propagation resulting in a lot of computation happening at compile time rather than run time.
Re: Outperforming LAPACK with C++ metaprogramming
#5It is somewhat unclear from his writing but did he compare this metaprogramming solution against OpenBlas or Atlas or just the reference BLAS/LAPACK implementation? If I was trying to do any significant linear algebra computations, I would definitely first try an optimized BLAS/LAPACK solution first.
Re: Outperforming LAPACK with C++ metaprogramming
#6[edit] I had a bug in my code; it does seem that the compiler is not picking up constants at compiletime. That is to say, it's not precomputing the final value at compiletime based on the value from those matrices.
Here's how I modified the code to make sure: https://pastebin.com/iG21pTiP
Re: Outperforming LAPACK with C++ metaprogramming
#7Please check with array initialized from file or other dynamic source. With maximal inlining, there's a risk of constant propagation resulting in a lot of computation happening at compile time rather than run time.
As an anecdote of just how powerful this can be, I once wrote a prototype of a simulation (written in C) that used a lot of integer divisions and needed to be run many times with different parameters.
As a quick and dirty hack that took about half hour to implement, I decided to just have the program as a template string and the parameters be substituted in by a python script that called GCC. To the amazement of a very sceptical friend, the result was a 5x speedup.
A lot of this speedup was consistent with replacing the integer divisions with multiplicative inverses, but also merging constants in a way that would have made things quite unreadable. I was quite impressed with how far GCC had gone.
Considering each instance of this simulation ran for tens of minutes, it was well worth compilation time of a few seconds. It kept the code really simple, too.
Re: Outperforming LAPACK with C++ metaprogramming
#8Please check with array initialized from file or other dynamic source. With maximal inlining, there's a risk of constant propagation resulting in a lot of computation happening at compile time rather than run time.
I think that's why it uses `volatile double` in stead of `double`?
Re: Outperforming LAPACK with C++ metaprogramming
#9Please check with array initialized from file or other dynamic source. With maximal inlining, there's a risk of constant propagation resulting in a lot of computation happening at compile time rather than run time.
[deleted]
Re: Outperforming LAPACK with C++ metaprogramming
#10It is somewhat unclear from his writing but did he compare this metaprogramming solution against OpenBlas or Atlas or just the reference BLAS/LAPACK implementation? If I was trying to do any significant linear algebra computations, I would definitely first try an optimized BLAS/LAPACK solution first.
Also, it seems the author solved a triangular system. LAPACK has special routines for that. Were they used?
LAPACK is typically optimized for larger systems, not 5 unknowns. Also, 5 is not a great number for vectorized operations - it might even be beneficial to zero/one pad the matrix.
Optimized LAPACK is often 5-10x faster than «basic LAPACK».
BLAS/LAPACK was originally written in the 70s/80s and sometimes unroll loops to the tune of 5 or 7 - it made sense then. Not so much these days.
I didn’t read the article in detail, but there appear to be a lot of holes in it.