Ensmallen – Flexible C++ library for efficient mathematical optimization

In terms of c++ matrix libraries, I've found eigen3 to be better than armadillo (although armadillo is easier to use).

Ceres is great for sparse, structured non-linear-least-squares problems (that is, problems where levenberg-marquardt is applicable). I believe it was initially designed for bundle-adjustment problems arising from structure-from-motion applications.

Ceres also does work for dense problems and general convex optimization, but it doesn't provide as much of an advantage over other packages (that is, you could just call into any existing l-bfgs implementation and wouldn't benefit much).

For some other problems, Ceres doesn't scale or can get in the way. For example, if you have a single value to maximize and a large number of variables, then ceres' auto-diff system won't work (ceres::Jet is based on "forward-mode" automatic differentiation, storing partial derivatives on the stack. If you have a high-dimensional gradient, it can exhaust stack space). On the other hand, Ensmallen doesn't seem to come with any auto-diff functionality.

This is looking very nice, can't wait to try it out. But I feel it's a missed opportunity to not provide some simple examples [1], as well as a benchmark to quickly estimate the amount of flops that can be expected. The main reason to use a library like this one over pyTorch or autograd/scipy is speed, after all.

[1] The test cases can be seen as examples, obviously, but they are not written in an easily accessible form IMHO.

EDIT: Looking more closely, it seems you need to provide an explicit gradient, no autodiff included, which makes it a complete nonstarter for me.

Huh, turns out you're right, which is entirely why em- and en- both exist and mean the same thing.

https://en.wiktionary.org/wiki/em-#Prefix