Live data from Hacker News

Algorithms for Optimization [pdf]

algorithmsbook.com

21–30 of 38 posts

Re: Algorithms for Optimization [pdf]

#21
post #16

Some additional optimization resources (for metaheuristics, where you only have the objective/score function and no derivative): - "Essentials of Metaheuristics" by Sean Luke https://cs.gmu.edu/~sean/book/metaheuristics/ - "Clever Algorithms" by Jason Brownlee https://cleveralgorithms.com/ Timefold uses the metaheuristic algorithms in these books (Tabu Search, Late Acceptance, Simulated Annealing, etc.) to find near-…

Timefold looks very interesting. This might be irrelevant but have you looked at stuff like InfoBax [1]? [1] https://willieneis.github.io/bax-website/

I haven't; from a quick reading, InfoBax is for when you have an expensive function and want to do limited evaluations. Timefold works with cheap functions and does many evaluations. Timefold does this via Constraint Streams, so a function like:

    var score = 0;
    for (var shiftA : solution.getShifts()) {
        for (var shiftB : solution.getShifts()) {
            if (shiftA != shiftB && shiftA.getEmployee() == shiftB.getEmployee() && shiftA.overlaps(shiftB)) {
                score -= 1;
            }
        }
    }
    return score
usually takes shift * shift evaluations of overlaps, we only check the shifts affected by the change (changing it from O(N^2) to O(1) usually).

That being said, it might be useful for a move selector. I need to give it a more in depth reading.

Re: Algorithms for Optimization [pdf]

#22
post #16

Earlier quoted context omitted.

Timefold looks very interesting. This might be irrelevant but have you looked at stuff like InfoBax [1]? [1] https://willieneis.github.io/bax-website/

I haven't; from a quick reading, InfoBax is for when you have an expensive function and want to do limited evaluations. Timefold works with cheap functions and does many evaluations. Timefold does this via Constraint Streams, so a function like: var score = 0; for (var shiftA : solution.getShifts()) { for (var shiftB : solution.getShifts()) { if (shiftA != shiftB && shiftA.getEmployee() == shiftB.getEmployee() && shi…

Thanks for the example. Yes, true, this is for expensive functions - to be precise functions that depend on data that is hard to gather, so you interleave the process of computing the value of the function with gathering strategically just as much data as is needed to compute the function value. The video on their page [1] is quite illustrative: calculate shortest path on a graph where the edge weights are expensive to obtain. Note how the edge weights they end up obtaining forms a narrow band around the shortest path they find.

[1] https://willieneis.github.io/bax-website/

Re: Algorithms for Optimization [pdf]

#23
An absolutely fantast book. I've read it cover to cover a couple of times during my PhD (which focused on neural networks and numerical solvers). Gives the right amount of depth to serve as a detailed introduction whilst covering a lot of the key areas in optimisation. I still use this book as a reference years later.

Re: Algorithms for Optimization [pdf]

#24

This book as well as Kochenderfer's earlier book "Decision Making Under Uncertainty"[0] are some of my favorite technical books (and I wouldn't be surprised to find his newest book, "Algorithms for Decision Making", also fell into this category). The algorithm descriptions are clear, the visualizations are great, and, as someone who does a lot of ML work, they cover a lot of (important) topics beyond just what is cov…

What other resources would you recommend for learning about optimizations? I use multi armed bandits a lot at work, and I wonder what other algorithms I should look into. Its hard to read the name of an algorithm or category and get a sense of whether or not its applicable. It seems like the general process of "which path out of N options is the best?" can be reframed in many ways depending on contextual details such…

Bertsekas.

Re: Algorithms for Optimization [pdf]

#26

This book as well as Kochenderfer's earlier book "Decision Making Under Uncertainty"[0] are some of my favorite technical books (and I wouldn't be surprised to find his newest book, "Algorithms for Decision Making", also fell into this category). The algorithm descriptions are clear, the visualizations are great, and, as someone who does a lot of ML work, they cover a lot of (important) topics beyond just what is cov…

What other resources would you recommend for learning about optimizations? I use multi armed bandits a lot at work, and I wonder what other algorithms I should look into. Its hard to read the name of an algorithm or category and get a sense of whether or not its applicable. It seems like the general process of "which path out of N options is the best?" can be reframed in many ways depending on contextual details such…

https://www.my-mooc.com/en/mooc/optimization-75993f57-f2df-4... https://www.coursera.org/learn/basic-modeling

Re: Algorithms for Optimization [pdf]

#28
Thanks for sharing this, looks like a useful overview of the field. I wish the first edition had come out a year or two earlier - this would have been a great resource for my undergrad research work. Back then there were no books covering CMA-ES, surrogate models, and gaussian processes all in one book, everything was scattered across different books and papers, with varying levels of technical depth and differing notations.

Re: Algorithms for Optimization [pdf]

#29
I'm astonished and also disappointed to see that the book has dedicated sections for the metaheuristics Firefly and Cuckoo Search. I don't know what happened there, but any experienced researcher from the field knows that these metaheuristics (among several others) are not serious and are very criticized in the community.

There is even a paper in ITOR about this: https://onlinelibrary.wiley.com/doi/abs/10.1111/itor.12001

Unfortunately, there are some "research bubbles" in the community of people that only work with these shady metaheuristics and keep citing each other. This is getting so bad to the point that I still frequently see in conferences people using such metaheuristics and believing that they are ok, only to get criticized by someone in the audience later.

Re: Algorithms for Optimization [pdf]

#30
post #19
post #4

Great to see optimization on the front page of HN! One thing I love about the book is it's full of really nice figures. If like me you love visualizations, you may enjoy this website I've been working on to visualize linear programming (LP) solvers: https://lpviz.net . It's by no means polished, but it can be pretty fun to play around with, visualizing how the iterates of different LP algorithms (described in section…

this is really brilliant!!

Thanks! I’m glad you enjoyed it. You may also get a kick out of the YouTube hyperlinks in the GitHub readme, especially the advanced methods for establishing convexity one :)
Post reply on HN