I can certainly believe that there is more going on than just stress, and the argument that compression waves travelling through the brain can damage it is pretty compelling. The question then is how would you block it? How would you protect yourself from a compression (blast) wave such that it went "around" your head rather than through it. Is there some material that could conduct that energy preferentially?
However, I'm on the side that thinks that periods of intense stress can produce a permanent, possibly physical, change in a person. These periods can be prolonged or particularly intense moments.