31–39 of 39 posts
Nocedal and Wright is an excellent overview and a good starting point. Also consider: - R. Schneider, Convex bodies: the Brunn-Minkowski theory. The first two chapters are an excellent introduction to convex geometry (plus a little bit extra!) if you have some undergrad-level analysis. - Hiriart-Urruty and Lemarechal, Fundamentals of convex analysis. This book has been highly recommended to me but I've never used it.…
https://www.amazon.com/Model-Building-Mathematical-Programmi... This is the canonical introduction book recommended by Gurobi. I've found it to be great for those getting into the subject. It has math of course, but the focus is on going over the basics of LP, integer, nonlinear, Mixed-Integer...etc, followed by lots of examples. I think it's the best book to start with to get a feel for the subject of OR, before div…
This is the cheat sheet for MIP that I wished I had when I started out. It lays out all of the most useful MIP formulations in a simple and compact way.
https://www.msi-jp.com/xpress/learning/square/10-mipformref....
(all generalizations are wrong to some extent and the delineations are not strict, but I have noticed they have different cultures -- analogous to how Breiman observed that there was two cultures to statistical modeling [1])
In the OR camp, the focus is primarily on linear programming/mixed integer linear programming. The types of problems solved include transportation, assignment, scheduling type problems. OR folks tend to go very deep into the theory of linear programs (matroids, Benders decompositions, etc.) and the literature is absolutely rich with advances in linear optimization. OR folks tend to be linear optimization specialists and mathematicians. A good practice-oriented intro book is Winston's "OR: Applications and Algorithms". Chvatal's Linear Programming is also good. But this is not my space, so I'll leave OR book recommendations to others.
In the Engineering camp, while folks do use linear programs, they also tend to go more in the direction of convex programs and general nonlinear/nonconvex programs (including mixed integer nonconvex nonlinear programs). There are some strong theoretical results for convex programs, but the results (naturally) aren't as strong for nonconvex problems -- global optimality never guaranteed. In the nonconvex camp, practitioners tend to concern themselves with heuristics/techniques like finding good initializations, understanding SSOCs, etc. The types of problems solved range from anything from ML problems to dynamic plant/machine optimization. Most folks in this camp tend to be engineers rather than optimization specialists (though some do become specialists eventually). Nocedal and Wright is a classic for general nonlinear programming, but also look into Biegler's Nonlinear Porgramming. Boyd and Vanderberghe is a classic for convex optimization. Murray and Gill's Practical Optimization is a bit outdated (so don't rely on it for state-of-the-art knowledge), but it has tidbits of insights about optimization that aren't found in other books and that continue to be useful.
https://www.amazon.com/Model-Building-Mathematical-Programmi... This is the canonical introduction book recommended by Gurobi. I've found it to be great for those getting into the subject. It has math of course, but the focus is on going over the basics of LP, integer, nonlinear, Mixed-Integer...etc, followed by lots of examples. I think it's the best book to start with to get a feel for the subject of OR, before div…
I used the Williams book in grad school and it provided context and motivation for MIP (mixed-integer) modeling, but for real-world MIP modeling, this guide from FICO is actually much better for practitioners. This is the cheat sheet for MIP that I wished I had when I started out. It lays out all of the most useful MIP formulations in a simple and compact way. https://www.msi-jp.com/xpress/learning/square/10-mipformr…
Mathematical optimization is huge field which splits into different branches. In my opinion, there are two demographics that approach optimization in slightly different ways and are interested in slightly different aspects of optimization: the OR (operations research) camp and the engineering camp. (all generalizations are wrong to some extent and the delineations are not strict, but I have noticed they have differen…
Earlier quoted context omitted.
I used the Williams book in grad school and it provided context and motivation for MIP (mixed-integer) modeling, but for real-world MIP modeling, this guide from FICO is actually much better for practitioners. This is the cheat sheet for MIP that I wished I had when I started out. It lays out all of the most useful MIP formulations in a simple and compact way. https://www.msi-jp.com/xpress/learning/square/10-mipformr…
Thanks for the tip!
y = a₀ + a₁ max(0, x − b₁) + a₂ max(0, x − b₂) + ...
max(0,x) functions are very easy to model in MIPs.
[1] https://en.wikipedia.org/wiki/Multivariate_adaptive_regressi...
Earlier quoted context omitted.
Thanks for the tip!
Just throwing another tip out there: if you're finding yourself having to model anything mildly nonlinear, but find yourself shackled by the linearity constraints of a Mixed Integer Program (MIP), you can use a statistical/ML technique called MARS [1] to regress a piecewise-linear model from your nonlinear data to approximate said nonlinear function. MARS models are essentially models with constant or max functions a…
I started from zero, and my approach was to read Nocedal/Wright cover-to-cover, and then the same with "Numerical Linear Algebra" by Trefethen/Bau. Usually it goes the other way, but I found the linear algebra primer in N/W to be good enough to get started. I also read "Practical Optimization" by Murray/Gill, which is interesting because it has a lot of conversational coverage of e.g. corner cases, stuff that most te…
So for medium to non-technical people reading this, I took a course in grad school that showed me how to do this in Excel with solver. It was easily one of the top 3 courses I took and heavily based off of this text book: https://www.amazon.com/Spreadsheet-Modeling-Decision-Analysi...
There are many programming libraries that would be better than Excel, e.g. scipy implements quite a few algorithms very accessibly, but that's not what OP is looking for.
I would recommend that text for anyone interested in getting into optimization. Excel quickly hits limits but the concepts translate to other tools