Live data from Hacker News

Ask HN: Do you use an optimization solver? Which one? Do you like it?

news.ycombinator.com

1–10 of 158 posts

Ask HN: Do you use an optimization solver? Which one? Do you like it?

#1
We’re trying to bring new ideas to the optimization space. We’ve used a bunch of technologies over time (CP, MIP, LP, heuristics, etc.). We’ve built our own solver (for https://www.nextmv.io) and learned a lot along the way.

Solvers are amazing. MIP solvers, for example, are some of the fastest and most mature software packages that exist. They’re also wickedly challenging to use effectively.

Do you use optimization solvers? Which ones? Why? Do you like them? Why or why not?

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#2
I would love to use one or more but the process to convert business logic to solver is painful so I ended up having to write a simulated annealing algo in Rust instead. I tried solver.com, Google OR-Tools, and a few other utilities.

It was much easier to build a score-calculator for min/max based on user-tweaked parameters, then, jiggle the data, re-calculate score, and keep doing it until there was significant improvement (again, standard SA). I would absolutely love to convert the entire production plan logic with material availability, lead-times, customer demand, quality control windows etc. to something like nextmv.io but looking at your docs, I have no idea where to begin.

Cost is a big factor too because 3 years ago I bought 4 old 24-core Xeons off eBay and they've been chugging non-stop simulating billion+ flops per hour with electricity being the only cost. I don't mind paying $50-100/day for cloud if the results are great and code is easy to manage. We have the same chicken-egg problem everyone in supply chain currently has - we don't have enough materials to make everything, don't know when we'll get everything, and so don't know the best order to buy/make everything in. I would love to write a solver for this using our dataset but I kind of don't want to re-invent the wheel.

As it stands, every solver I find is one layer of abstraction away from what I want to code in. I can explain the problem in length if you want but it's honestly nothing unique - just the standard MRP/ERP planning with ton of BOM items, PO delays, labor/machine capacity constraints etc.

Your existing tutorials explain how I can use your API/SDK to perform OR operations. That's great and a necessity. However, it's not sufficient for me because my questions are: How do I represent my production calendar in the JSON blob for your algo? How do I put a constraint of 100hrs/week/machine but also 168hrs/week/room full of specific machines. In other words, while each machine can run 100hrs/week, if there are 4 of them in the same room, only one can run at a time, and so combined machines in a given room cannot be over 168hrs/week. Maybe a tutorial or a higher-level library to help people like me convert business rules into JSON format for your APIs. Because even if I might be capable of using your API as-is, I unfortunately don't have the time to implement these things myself. Hope this makes sense and gives you some insight into at least one of your target use-cases.

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#3
post #2

I would love to use one or more but the process to convert business logic to solver is painful so I ended up having to write a simulated annealing algo in Rust instead. I tried solver.com, Google OR-Tools, and a few other utilities. It was much easier to build a score-calculator for min/max based on user-tweaked parameters, then, jiggle the data, re-calculate score, and keep doing it until there was significant impro…

Thanks for looking at our docs in light of your problem domain. That's way beyond what I anticipated, and much appreciated!

We haven't put a lot more examples into our SDK docs lately since we've been working more on our routing engine and cloud offering. Now we're getting back to more foundational questions like "what should our solver be?" and "how should model formulation work?"

Hop started off as a decision diagram solver, but even internally we strap a lot of different techniques onto it. My hope is to support those patterns better, which is really why I posed this question.

I'd be interested to know: what made the process of converting business logic into solver-speak painful?

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#4
I use JuMP as modeling language. For MILP i am usually using Gurobi or SCIP. For ILP problems have have been looking in to the exact solver https://gitlab.com/JoD/exact which seems quiet promising.

For NLP i usually go with either https://worhp.de/ or just IpOpt.

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#5
post #2

I would love to use one or more but the process to convert business logic to solver is painful so I ended up having to write a simulated annealing algo in Rust instead. I tried solver.com, Google OR-Tools, and a few other utilities. It was much easier to build a score-calculator for min/max based on user-tweaked parameters, then, jiggle the data, re-calculate score, and keep doing it until there was significant impro…

Thanks for looking at our docs in light of your problem domain. That's way beyond what I anticipated, and much appreciated! We haven't put a lot more examples into our SDK docs lately since we've been working more on our routing engine and cloud offering. Now we're getting back to more foundational questions like "what should our solver be?" and "how should model formulation work?" Hop started off as a decision diagr…

> what made the process of converting business logic into solver-speak painful?

The fact that every solver doc I found shows me two near identical variables and 4 identical constraints. I can solve that using Excel Add-In. My problem is mind-numbingly complex, like everyone else doing supply-chain optimization. So I need examples for each type of constraint I have but instead, the tools expect me to figure out everything based on very generic examples. Dates are a big deal in what I do. I have no idea how to convert "each day the customer order is delayed is bad, but I also can't make things 3 months before they want it" into solver-speak. Cleaning a machine takes time, so it is often better to make similar things in a row (campaign-mode). No idea how to convert that to solver-speak.

The good thing is that once I understand how to convert my data into solver-speak, I can keep it updated on the fly since business logic doesn't change daily, just the data-set.

Feel free to contact me directly and/or we can Zoom etc. if you wish to discuss further. Wish you the best of luck either way!

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#6
post #5

Earlier quoted context omitted.

Thanks for looking at our docs in light of your problem domain. That's way beyond what I anticipated, and much appreciated! We haven't put a lot more examples into our SDK docs lately since we've been working more on our routing engine and cloud offering. Now we're getting back to more foundational questions like "what should our solver be?" and "how should model formulation work?" Hop started off as a decision diagr…

> what made the process of converting business logic into solver-speak painful? The fact that every solver doc I found shows me two near identical variables and 4 identical constraints. I can solve that using Excel Add-In. My problem is mind-numbingly complex, like everyone else doing supply-chain optimization. So I need examples for each type of constraint I have but instead, the tools expect me to figure out everyt…

"My problem is mind-numbingly complex, like everyone else doing supply-chain optimization" brings me joy. I'll definitely reach out - would love to get your reaction to some of our new ideas.

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#7
post #4

I use JuMP as modeling language. For MILP i am usually using Gurobi or SCIP. For ILP problems have have been looking in to the exact solver https://gitlab.com/JoD/exact which seems quiet promising. For NLP i usually go with either https://worhp.de/ or just IpOpt.

Hadn't heard of JoD - that looks really interesting. What are your experiences so far?

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#8
SCIP, GLPK.

I don't mind the overhead in developer time. Once you 'git gud' at translating problems to MIP/ LP, it's just a matter of learning syntax. The real trick is in translating problems.

It's a shame more CS courses don't focus on this problem translation meta-algorithm. You can get it in tough theory courses with problem mapping to prove complexity, but its usefulness goes way, way beyond that somewhat niche application. Learning to model problems as max flow, graph partitioning / matching, LP, MIP, gives you absolute super powers way, way beyond what you'd gain by having a solver that's easier to work with.

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#9
post #4

I use JuMP as modeling language. For MILP i am usually using Gurobi or SCIP. For ILP problems have have been looking in to the exact solver https://gitlab.com/JoD/exact which seems quiet promising. For NLP i usually go with either https://worhp.de/ or just IpOpt.

Hadn't heard of JoD - that looks really interesting. What are your experiences so far?

JoD is Jakob Nordström. He gave a talk on the predecessor of Exact here: https://www.youtube.com/watch?v=yvXy7Nzn-IA

My experience so far: Quiet good at finding feasible solutions when constraints are complex. Really young code with a lot of potential. Not that good at finding optimas.

Re: Ask HN: Do you use an optimization solver? Which one? Do you like it?

#10
post #2

I would love to use one or more but the process to convert business logic to solver is painful so I ended up having to write a simulated annealing algo in Rust instead. I tried solver.com, Google OR-Tools, and a few other utilities. It was much easier to build a score-calculator for min/max based on user-tweaked parameters, then, jiggle the data, re-calculate score, and keep doing it until there was significant impro…

Yes, translating the problems is still the hardest part. Once you can write down the MIP, it's just syntax to get it into any old solver. It took me two - three days to write the I/O to get data in, formatted properly, and so on, and about half a day to write the MILP to solve ship-design optimization in Highfleet.

https://github.com/jodavaho/highfleet-ship-opt

Post reply on HN