I think it depends on what you mean by "you can afford it".
At some level that's the nature of midlife crisis, identifying what it is you have and what you want, and what you can have.
If you are happy with what you have despite all its costs, it's not really a crisis because you've identified that as such. Even if you're not entirely happy with it but you recognize that it is a net good for you and your family, there's no crisis either. If some things you identified as benefits before are now seen as serious costs — to family, children, morality, or integrity, among other things — but there's an easy and painless way to rectify things, there's also no crisis.
The problem — the crisis — is in recognizing or believing that what you once saw as good things are actually costly, but rectifying it is also fraught with large and probably unknown costs. Maybe you've realized the vocation you're in is actually rife with immorality and corruption, or maybe you've realized that your career has completely cost your children precious, limited time with a good family, or whatever, but you've gotten so far along that the costs and likelihood of changing careers are also great. That's the nature of the crisis.
At some level I think you could say it's a crisis of privilege but I don't think so. I think it comes anytime a set of life choices, over a long time and with many costs, leads one to a position of deep dissatisfaction and realization that there's limited time to right things. My sense is that doesn't really depend on socioeconomic status necessarily.
The other side of it is there are apparently studies showing similar mood changes over the course of life happen in completely other species, so there's an argument that it really has nothing to do with existential or sociological issues at all and is very biological. Based on my own experiences that seems hard to believe though; my guess is midlife crises can happen for many different reasons, and aren't limited to midlife.