Black box optimization competition
bbcomp.ini.rub.de
Black box optimization competition
1–10 of 26 posts
Re: Black box optimization competition
#2Seems really interesting. Too mathy for my skillset.
If I may, I propose that the organizers remove the restriction on disassembling the client library or intercepting network connections. This restriction seems like it cannot benefit the organizers, unless the protocol is insecure. People are going to ignore this rule anyway, and you can't stop them or even detect them doing it. So why put it in there? It's only going to generate ill will.
Re: Black box optimization competition
#3Wish I had seen something about this sooner. The competition began in January and ends on the 30th of this month.
Re: Black box optimization competition
#4Wish I had seen something about this sooner. The competition began in January and ends on the 30th of this month.
There are two tracks, the second has the deadline in the end of June.
Re: Black box optimization competition
#5I really dislike the term "Black box optimization". There's no such thing. You have to make assumptions about your function, so in the end this is just rewarding people whose optimizers happen to match the chosen functions; but those functions are not made explicit whatsoever. That doesn't make any sense.
For example, if the output/input are floating point numbers than you can assume the domain/range is [-M,M]. Otherwise, with even the most clever function you have no guarantee of ever approaching the optimum, even if the function is continuous. Now even with a limited range there are no guarantees if the function is not well behaved -- so you have to again assume the function is well behaved. And for any assumption you make there is a condition on function for which it is terrible. There is no best assumption, or best algorithm, then. You could, for instance, assume the function is adversarial (trying to make your life difficult), for which the best algorithm is perhaps just sampling randomly the range, which is really a terrible algorithm -- but that's of course just another assumption, and a terrible one.
I would much prefer 'Typical function optimization', if you're optimizing unlabeled functions so frequently, or at least not try to hide the inevitable assumptions.
TL;DR: The contest may be useful, but the concept of "Black box optimization" is nonsense.
Re: Black box optimization competition
#6I really dislike the term "Black box optimization". There's no such thing. You have to make assumptions about your function, so in the end this is just rewarding people whose optimizers happen to match the chosen functions; but those functions are not made explicit whatsoever. That doesn't make any sense. For example, if the output/input are floating point numbers than you can assume the domain/range is [-M,M]. Other…
The domain is the unit cube [0, 1]^d using double precision floating point, see the documentation.
Making assumptions and testing them is very much part of the contest. You are even allowed to do this interactively.
Re: Black box optimization competition
#7Seems really interesting. Too mathy for my skillset. If I may, I propose that the organizers remove the restriction on disassembling the client library or intercepting network connections. This restriction seems like it cannot benefit the organizers, unless the protocol is insecure. People are going to ignore this rule anyway, and you can't stop them or even detect them doing it. So why put it in there? It's only goi…
It's probably insecure, because you don't want to do 75,850,000 sequential evaluations over a network. It would take over a week for a single track with even just 10ms response time.
Re: Black box optimization competition
#8Seems really interesting. Too mathy for my skillset. If I may, I propose that the organizers remove the restriction on disassembling the client library or intercepting network connections. This restriction seems like it cannot benefit the organizers, unless the protocol is insecure. People are going to ignore this rule anyway, and you can't stop them or even detect them doing it. So why put it in there? It's only goi…
It's probably insecure, because you don't want to do 75,850,000 sequential evaluations over a network. It would take over a week for a single track with even just 10ms response time.
Presumably a secure implementation would just batch up evaluations into groups of ten or a hundred. You could make something like this work.
Re: Black box optimization competition
#9Whoa. The servers for this competion are about 8km away. That's the most 'local' content I've ever seen on HN.
Unfortunately I have to agree with obstinate here. The pure math is too much for me and reverse engineering (still daunting, but interesting/possible) is not acceptable. If any HN person wins this contest, I offer beers close to the black box :)
Re: Black box optimization competition
#10It's a little strange that they do not have a track that gives gradient information, given that it is often a real world possibility. Also, this basically allows unlimited time between eval... So this becomes a contest about
- coming up with a distribution over R^n -> R function
- finding the optimal evaluation points to do Bayesian update
I predict the winner will use some a mixture of Gaussian processes with various kernels and stochastic control (with a limited look ahead, otherwise it blows up) to pick the test points.