Ask HN: Literature for mathematical optimization?

So for medium to non-technical people reading this, I took a course in grad school that showed me how to do this in Excel with solver.

It was easily one of the top 3 courses I took and heavily based off of this text book:

https://www.amazon.com/Spreadsheet-Modeling-Decision-Analysi...

Sounds like you're looking more for optimization theory, but if you want a gentle introduction to applications with approachable math and lots of examples, I highly recommend "Operations Research: Applications and Algorithms (4E)" by Wayne Winston. It's a solid undergrad level text covering basic linear optimization, mixed integer linear programs, and non-linear optimization.

Papadimitriou was not Gates's MSc thesis supervisor. Gates never did an MSc (nor indeed completed his BSc -- he dropped out of college to found some computer company or other).

What did happen is that Papadimitriou and Gates were co-authors of a paper about pancake sorting. (https://en.wikipedia.org/wiki/Pancake_sorting)

Nocedal and Wright is an excellent overview and a good starting point. Also consider:

- R. Schneider, Convex bodies: the Brunn-Minkowski theory. The first two chapters are an excellent introduction to convex geometry (plus a little bit extra!) if you have some undergrad-level analysis.

- Hiriart-Urruty and Lemarechal, Fundamentals of convex analysis. This book has been highly recommended to me but I've never used it. Might be an easier go than Schneider for convex geometry.

- Golub and van Loan, Matrix computations. Excellent book on numerical linear algebra.

- Bonnans, Gilbert, Lemarechal, Sagastizabal, Numerical optimization: theoretical and practical aspects. This book has a detailed description of bundle methods, which are important and in my opinion underutilised.

- I. Maros, Computational techniques of the simplex method. This is the only book I'm aware of that discusses how to build a working implementation of the simplex method for linear optimisation.

I'm not aware of any books that cover line search algorithms in detail. These are important in implementations but, beyond discussing the Goldstein and Wolfe conditions, generally glossed over in prose. Even in the absence of stalling and numerical difficulties, you can see an order of magnitude speedup from replacing a bad line search with a good one. One line search algorithm I've had success with is described in More and Thuente, Line search algorithms with guaranteed sufficient decrease.

Lots of tacit engineering knowledge goes into building a fast and robust optimisation code. Some of that knowledge gets forgotten when code is rewritten or ported from one language to another.

Mercifully, a lot of that engineering knowledge has been encoded into freely-available optimisation code. Quite a bit of that code is pretty readable. Off the top of my head, I've learnt things from:

- Liu and Nocedal's Fortran L-BFGS implementation,

- The CUTEst problem collection,

- Chih-Jen Lin's LIBLINEAR and LIBSVM,

- Lin and More's TRON,

- Csaba Meszaros's BPMPD,

- Jacek Gondzio's HOPDM,

- The GNU Linear Programming Kit,

- and probably quite a few other sources!

He said "your #1", making reference to the first book in OPs list: "Numerical Optimization" by Jorge Nocedal and Stephen J. Wright.

It's pretty fun to read.