11–20 of 45 posts
An aside, for things like unit tests where in most cases the result of the test is binary i.e fail/pass. How do you think about and ultimately represent the fitness criteria.
A long time ago I had huge interest in genetic algorithms and on other soft optimization techniques. It is what inspired me to take up graduate studies in the first place. But very quickly I got thoroughly disillusioned by the community around it. What I am going to say is going to be very unpopular to the audience of this post. What turned me off and left a bad taste is the tendency for the community to push it as m…
In my opinion, EAs are most useful when you don't have a more specific human-fit algorithm to solve a problem, and particularly when a gradient to your objective function cannot be calculated. For example, when you are trying to create a controller for a many-jointed robot in a complex simulation with a high-level objective function.
Earlier quoted context omitted.
That sounds like over fitting. It happens with all optimization routines, is the EA somehow special in this regard?
Overfitting is when your model is over-fit to the data you are training on [1], but what we're talking about here is slightly different -- the EA has maximized the given objective function correctly , but it turns out that your objective function is corruptible: A high value on your simple measure doesn't match the intuition motivating it. You had something particular in mind when you set the objective function for y…
if validation were performed the population would not evolve into the cracks of unlikely physics loop holes.
I highly recommend the hard copy
A long time ago I had huge interest in genetic algorithms and on other soft optimization techniques. It is what inspired me to take up graduate studies in the first place. But very quickly I got thoroughly disillusioned by the community around it. What I am going to say is going to be very unpopular to the audience of this post. What turned me off and left a bad taste is the tendency for the community to push it as m…
I'm not sure that the phenomenon you've encountered is unique to the evolutionary computation community: If you go to an academic conference dedicated to specific methodology, because the attendees have a vested interest in that methodology (they may have built their career on it), that hammer will certainly end up finding many questionable 'nails'. In my opinion, EAs are most useful when you don't have a more specif…
From my experience, and this was a while ago so I will be glad if this has changed, it seems that the community prefers to push their techniques as snake oil and not try to nail down the characteristics of their techniques and show how to match it with a function I want to optimize. I want them to offer principled guidelines that would allow generalizing the techniques beyond anecdotes. I have no problems with communities pushing their technique, to the contrary, my disenchantment is with them not doing this.
I want the community to produce re-usable pieces of interesting/novel information that I can use when I am faced with optimizing a function.
You mentioned non-differentiable functions. Now lets take a look at a subclass of these functions: convex non-differentiable functions. There are very efficient methods for these.
Consider another class, lets throw away convexity, consider functions that are non-differentiable and very rough locally but when filtered with a low pass filter is well behaved. Then again we know what to do.
Consider functions that are difference of potentially non differentiable convex functions (this is a Huge class. The difference need neither be differentiable, nor be convex), then again we have good ideas about what to do.
I think building this decision function: Problem_type -> preferred_algorithm is a very useful exercise. What annoys me is that GA community seems not to be interested in this, and take cheap shots by presenting anecdotes.
Prove properties, of your techniques, I will buy them by the bagful.
Earlier quoted context omitted.
That sounds like over fitting. It happens with all optimization routines, is the EA somehow special in this regard?
Overfitting is when your model is over-fit to the data you are training on [1], but what we're talking about here is slightly different -- the EA has maximized the given objective function correctly , but it turns out that your objective function is corruptible: A high value on your simple measure doesn't match the intuition motivating it. You had something particular in mind when you set the objective function for y…
So those are "overfitting" for the exact quirks of the engine. I agree, it's a stretch, but not unlike in ML, it shows that optimizing too much will show quirks if your 'objective function/simulation' pair is not exactly equal to the 'desired objective function/reality', or if there is an innumerable ensemble of situations you might expect the solution to behave well.
Earlier quoted context omitted.
There are people using "generated" tests based on specs crafted by programmers. The whole thing is called "generative" testing. The reason why this stuff works, is that there are almost always lots of bugs out there, and that RNGs aren't subject to the misconceptions programmers are subject to. It's also why fuzzing works. I could see GAs doing very well with this.
I definitely think that GAs can uncover a variety of bugs ranging from simple NPEs and more nuanced bugs like memory leaks (which would then required human dev intervention to investigate for a post-mortem) by dynamically generating the test-inputs. In addition to simply generating the input data, do you feel like GAs could broaden their span to essentially "mock" the states of other components in the system? I'm thi…
Skiena in The Algorithm Design Manual, at the end of the three-paragraph blurb where he talks about them.