Live data from Hacker News

Intro to Multicriteria Optimization

blog.sigopt.com

21–30 of 46 posts

Re: Intro to Multicriteria Optimization

#21

> The reason this is a more complicated situation is that an ordering of vectors in RkRk does not exist... how would one order the vectors u = (1,2,3), v = (2,1,3), w = (3,2,1)? It seems straightforward to order those vectors by first comparing the first component, then the second, then the third. The result is u,v,w. It's as if you wanted to sort a multi-column report in Excel. What am I missing?

Consider a simpler case of two dimensional real vectors. Lets consider the vector space of R^2, since all finite dimensional vector spaces of same dimensionality are isomorphic, R^2 is isomorphic to the complex numbers. How would you order the complex numbers?

> How would you order the complex numbers?

You can use any order that can be defined on R^2. It's just that you can show that this order cannot satisfy the axioms of an ordered field.

Re: Intro to Multicriteria Optimization

#22
post #16

Earlier quoted context omitted.

I do research in combinatorial optimization. Does your company solve any problems with a combinatorial flavor? (say, things can be optimized using combinatorial algorithms instead of going for gradient descent.

We don't (yet), but we are growing rapidly and always looking for ways to help our customers solve optimization problems. Fwiw, we do support categorical parameters (as well as continuous and integer) and our ensemble of Bayesian optimization techniques are able to solve this mixed type problem much more efficiently than techniques like gradient decent. Although the way we handle the purely combinatorial (only catego…

> [We] are able to solve this mixed type problem much more efficiently than techniques like gradient decent.

Naive gradient descent is probably the simplest strategy that one can imagine. How do your algorithms compare to Newton methods for minimizing the primal-dual gap (interior point methods)?

Re: Intro to Multicriteria Optimization

#23
For anyone looking for a free optimization tool in python, sypy.opimize is easy to use.

Eg if I have a complicated function revenue([A,B,C,D])

I can define obj([A,B,C,D]) = -1* revenue([A,B,C,D])

and use:

  >>>import numpy as np

  >>>from scipy.optimize import minimize

  >>>X0 = np.array([1.5, 0.7, 1.2, 100])

  >>>options={'xtol': 1e-4, 'disp': True})

  >>>X* = minimize(obj, x0, method='nelder-mead', options)
http://docs.scipy.org/doc/scipy/reference/tutorial/optimize....

Re: Intro to Multicriteria Optimization

#24
post #23

For anyone looking for a free optimization tool in python, sypy.opimize is easy to use. Eg if I have a complicated function revenue([A,B,C,D]) I can define obj([A,B,C,D]) = -1* revenue([A,B,C,D]) and use: >>>import numpy as np >>>from scipy.optimize import minimize >>>X0 = np.array([1.5, 0.7, 1.2, 100]) >>>options={'xtol': 1e-4, 'disp': True}) >>>X* = minimize(obj, x0, method='nelder-mead', options) http://docs.scipy…

scipy.optimize can only do single-criterion optimization (scalar-valued objective function instead of vector-valued objective functions).

Re: Intro to Multicriteria Optimization

#25
post #13

Earlier quoted context omitted.

> most people would be willing to arrive 0.0000001 hours later to save 998 dollars Unfortunately, if you don't know how your model behaves, you can't tell when you're setting γ whether you're actually going to get a solution that's way past the point of diminishing returns. You may not even be able to tell after the fact. This is one of the reasons for doing sensitivity analysis on γ. (Your ε-constraint scalarization…

You make a perfectly accurate point that, in practice, it is unlikely one would be able to make such a prediction without significant info about the model. The above comment was meant in more of a post hoc "we've executed our multicriteria optimization, approximated our Pareto frontier, now let's make a decision" sense, not within a specific scalarization context. Indeed sensitivity analysis on the gamma parameter is…

I don't have any references -- I'm going off something one of my committee members said. But the basic idea is just that, if your weight is 0.2, you want to make sure you also obtain solutions for 0.18 and 0.22, to give you some sense of whether you're on the edge of a cliff. This is pretty cheap in general for convex optimization problems because you've already got a solution that should be close to optimal for the new weight.

Re: Intro to Multicriteria Optimization

#26
post #21

Earlier quoted context omitted.

Consider a simpler case of two dimensional real vectors. Lets consider the vector space of R^2, since all finite dimensional vector spaces of same dimensionality are isomorphic, R^2 is isomorphic to the complex numbers. How would you order the complex numbers?

> How would you order the complex numbers? You can use any order that can be defined on R^2. It's just that you can show that this order cannot satisfy the axioms of an ordered field.

[deleted]

Re: Intro to Multicriteria Optimization

#27
post #12

Earlier quoted context omitted.

Does your platform support more than one level of preemption / lexicographic goal setting? (Like the ε-constraint scalarization technique, but with a second, third, nth set of goals between the first set and the objective?)

Our customers who are working with multicriteria problems have, thus far, had primarily two criteria, thus we have been helping them manage their two criteria problems into a scalar setting. As such, we do not, at this moment, permit the layers of ordering strategy you suggest through our API. To do so internally would introduce a complicated bifurcation between problems phrased with real-valued observations (as is o…

I agree that expressing the problem can become more confusing in the presence of more levels of preemption, however it can be an effective way to organize large numbers of criteria. If your customers are mainly working with biobjective problems, I can see why you wouldn't be too interested in adding more levels!

Re: Intro to Multicriteria Optimization

#28
post #24
post #23

For anyone looking for a free optimization tool in python, sypy.opimize is easy to use. Eg if I have a complicated function revenue([A,B,C,D]) I can define obj([A,B,C,D]) = -1* revenue([A,B,C,D]) and use: >>>import numpy as np >>>from scipy.optimize import minimize >>>X0 = np.array([1.5, 0.7, 1.2, 100]) >>>options={'xtol': 1e-4, 'disp': True}) >>>X* = minimize(obj, x0, method='nelder-mead', options) http://docs.scipy…

scipy.optimize can only do single-criterion optimization (scalar-valued objective function instead of vector-valued objective functions).

> The tradeoffs between these two criteria can either be managed by some supervisory decision maker (the driver of the car in this example) or by merging the multiple criteria into some single criteria and phrasing the problem as a standard optimization problem.

You always have to reduce the problem to a scalar if you want a single answer.

Re: Intro to Multicriteria Optimization

#29
post #24
post #23

For anyone looking for a free optimization tool in python, sypy.opimize is easy to use. Eg if I have a complicated function revenue([A,B,C,D]) I can define obj([A,B,C,D]) = -1* revenue([A,B,C,D]) and use: >>>import numpy as np >>>from scipy.optimize import minimize >>>X0 = np.array([1.5, 0.7, 1.2, 100]) >>>options={'xtol': 1e-4, 'disp': True}) >>>X* = minimize(obj, x0, method='nelder-mead', options) http://docs.scipy…

scipy.optimize can only do single-criterion optimization (scalar-valued objective function instead of vector-valued objective functions).

True, you need to assign weights to a vector of outputs as discussed in this article. After that, scipy.optimize is easy to use though.
Post reply on HN