Eigen: A C++ template library for linear algebra

On the other hand, Eigen didn't claim to be faster than any BLAS during my interaction with informational side of their website. Nowadays, I just get the latest version, update my code and directly reach for their reference guide.

However, being used by both CERN (in LHC ATLAS) and TensorFlow, I guess they're no slouch in terms of performance (it also genuinely amazed my heavy BLAS fan professor at my Jury when the performance numbers came up). Another plus is, Eigen's ability to be used in CUDA kernels (though I didn't try it yet).

On the other hand, they're insanely fast (almost BLAS fast in my experience) for what I'm doing, and it's really well optimized (shows when you use -O3, especially on the solvers side). Also, it has neat capabilities like dynamic matrices and instant submatrix access (you can get a submatrix of a matrix, or just patch in a submatrix in a single call).

About RAM placement: From my experience, code becomes overly dependent on non-fragmented memory when allocating big matrices and/or vectors naively (i.e. in a contiguous manner), and execution fails as the node you're running on (or your computer) gets busier with other code. Eigen counters this by intelligently dividing bigger matrices into smaller chunks, both optimizing the access time and reducing (or effectively eliminating from my experience) the risk of segmentation faults related to failed memory allocations due to not having not enough undivided space in the memory space.

For example, we had simulation test suites which would run ~30 seconds of real-world time. If we can run those sims at 10x real time, each only takes 3 seconds, great! But with optimizations turned off, heavy Eigen code would run 1000x slower, which means the sims now take nearly 3000 seconds to run when bug hunting in the non-Eigen related surrounding logic. This already makes the test suite virtually un-runnable in debug builds, but if you want to then run those binaries through Valgrind (which tacks on another 100-1000x performance tax) you're now talking about something which is completely intractable. You can't run optimized builds through Valgrind without potentially introducing false positives, so you can't even run Valgrind on only the optimized builds... you just can't run it at all because your linear algebra library is slowing everything down.

The core problem is Eigen's heavy use of expression templates, which the optimizer cuts through just fine. But without the optimizer turned on, it adds layers and layers and layers of unnecessary abstraction. Because these are all templates, C++ expands all this code at every call site, making it substantially less efficient and impossible to shim out by linking with an optimized build of Eigen itself.

The only approach I've seen get close to making this work was an attempt by an engineer to use explicit template instantiation to create a separate binary with all the necessary Eigen functions which could be compiled separately with the optimizer turned on. However, this proved too nightmarish because since Eigen can template on the dimensions of everything, you need to know up front exactly all the various instantiations of Eigen types the program uses. This might be doable by writing a clang plugin or something, but doing it by hand was just way too much work because the codebase was huge, so we couldn't justify spending more time on it.