I've found that intentionally causing abrupt (but reasonable) changes to hyperparameters in evolutionary spiking neural network simulations (I.e between each generation) results in far more robust simulations that will meet fitness criteria with less likelihood of getting stuck somewhere. The tradeoff being that simulating will take longer, but this may be worth things like reducing the chances of your resource requi…
Very cool, but I'm surprised you're sharing this on here and not in a job interview with a deep learning startup and/or an arXiv paper.
Respect to the OP as spiking networks are a different domain, but the field of Graceful Degradation and Perturbations itself is almost older than the field of Neural Networks itself. Generally, the bitter lesson occurs - these techniques show some kind of useful robustness at parameter M, but fail to show as much of a useful robustness relatively as parameter M*100. For example, before attention, using Drop out (in essence a form of forcing robustness) was one of the most common techniques. The quality of robustness or its resulting sub parts (e.g. the tradeoff between precision and recall) is incorporated as a measure in some of the fields basic scores, such as harmonic F1 score.
Still it, like alternate activation functions, data beyond matrices and vectors and non-linear networks, remain very unstudied.