Unexpected Benefits of Self-Modeling in Neural Systems

This is a nice regularization method, and I'm curious how it compares to model distillation as a simplification method. Does it do better? Is it synergetic or redundant with a distillation approach? Does it simplify in a way that also makes it more human-intelligible?