Ask HN: Literature for mathematical optimization?

#1

Post by samuel2 » Sat, Jul 03, 2021, 7:16 AM UTC

Hi! Do you know any books on math optimization that are essential for anybody getting into this field? Is there any classical literature for optimization related to ML? Thanks!

My current list includes:

1. Numerical Optimization by Jorge Nocedal Stephen J. Wright

2. Algorithms for Optimization - introduction to optimization with a focus on practical algorithms

3. Algorithms for Decision Making - a broad introduction to algorithms for decision making under uncertainty

[2] https://algorithmsbook.com/optimization/

[3] https://algorithmsbook.com/

Re: Ask HN: Literature for mathematical optimization?

#2

Post by thxg » Sat, Jul 03, 2021, 8:12 AM UTC

For theoretical continuous/nonlinear/convex optimization, your #1 is the bible, together with

"Convex Optimization" by Boyd & Vandenberghe.

However, beware that both are grad textbooks. They can be tough going at times. Unfortunately, I never found undergrad textbooks I liked much, for theory.

If you're interested in discrete optimization too (the other half of math optimization), the classics are:

"Optimization Over Integers" by Bertsimas & Weismantel

"Integer and Combinatorial Optimization" by Nemhauser & Wolsey

Re: Ask HN: Literature for mathematical optimization?

#3

Post by samuel2 » Sat, Jul 03, 2021, 8:18 AM UTC

thxg wrote: post #2

For theoretical continuous/nonlinear/convex optimization, your #1 is the bible, together with "Convex Optimization" by Boyd & Vandenberghe. However, beware that both are grad textbooks. They can be tough going at times. Unfortunately, I never found undergrad textbooks I liked much, for theory. If you're interested in discrete optimization too (the other half of math optimization), the classics are: "Optimization Over…

Thank you a lot!

Re: Ask HN: Literature for mathematical optimization?

#4

Post by brudgers » Sat, Jul 03, 2021, 4:36 PM UTC

Two fascicles for Knuth's Art of Computer Programming, volume 4B on optimization are out. https://www-cs-faculty.stanford.edu/~knuth/news.html

Re: Ask HN: Literature for mathematical optimization?

#5

Post by gradschool » Sat, Jul 03, 2021, 7:14 PM UTC

"The Design of Approximation Algorithms" by Williamson and Shmoys

http://www.designofapproxalgs.com/

Re: Ask HN: Literature for mathematical optimization?

#6

Post by chronal » Sun, Jul 04, 2021, 5:42 AM UTC

samuel2 wrote: post #3

thxg wrote: post #2

For theoretical continuous/nonlinear/convex optimization, your #1 is the bible, together with "Convex Optimization" by Boyd & Vandenberghe. However, beware that both are grad textbooks. They can be tough going at times. Unfortunately, I never found undergrad textbooks I liked much, for theory. If you're interested in discrete optimization too (the other half of math optimization), the classics are: "Optimization Over…

Thank you a lot!

You grab a copy of "Convex Optimization" by Boyd & Vandenberghe over here: https://web.stanford.edu/~boyd/cvxbook/

This is more optimal control but I really enjoyed reading through these notes: https://math.berkeley.edu/~evans/control.course.pdf

Re: Ask HN: Literature for mathematical optimization?

#7

Post by samuel2 » Sun, Jul 04, 2021, 11:46 AM UTC

gradschool wrote: post #5

"The Design of Approximation Algorithms" by Williamson and Shmoys http://www.designofapproxalgs.com/

cool, thanks

Re: Ask HN: Literature for mathematical optimization?

#8

Post by eigenman » Sun, Jul 04, 2021, 2:56 PM UTC

I'd also add C. T. Kelley's "Iterative Methods for Optimization" for more non convex theory. Nemirovski also has a variety of books and course notes that are available, but I haven't spent as much time with them.

I agree with thxg, there are few undergraduate textbooks that I've liked.

Re: Ask HN: Literature for mathematical optimization?

#9

Post by nknealk » Sun, Jul 04, 2021, 3:02 PM UTC

I recommend Bayesian Methods for hackers [1]. It doesn’t go too deep on theory but I feel like it’s well written and has really good coded up examples of theory being applied to problems. It has a relatively narrow scope, but I find myself reaching for methods I learned from the author frequently.

[1] https://github.com/CamDavidsonPilon/Probabilistic-Programmin...

Re: Ask HN: Literature for mathematical optimization?

#10

Post by jthickstun » Sun, Jul 04, 2021, 3:04 PM UTC

Sébastien Bubeck's book is excellent mathematical introduction to modern convex optimization: https://arxiv.org/abs/1405.4980