For example, some commonly associated traits ascribed to narcissism are a desire of power, status, or admiration from others; the the self-perception of being unique, and a lack of empathy for others. Now why would someone psychologically end up fixated on these traits? Well perhaps, they've been treated poorly and are dejected from others. They are on the receiving end of lack of empathy, are downtrodden, and abused. If they had power, status, admiration, or generally were treated well they wouldn't feel like their ego's were so fragile. So they exaggerate their importance to try to gain what they're lacking, and lash out if they feel they are wronged.
So, to me, people with these traits seem to be concerned about what others think of them. Sure the "them", at the end of that last sentence makes it seem they're being self-centered. But read it again; the words "concerned about others" co-exist in that sentence - words that describe altruism. Why would someone be concerned with their appearance? Perhaps they've been treated poorly by others... and are worried they're unacceptable to society... So they focus on themselves.
I'd say don't worry about it much, it's easy to beat yourself up. I know that's not much help, but for me thinking this way helps me realize I'm not perfect nor no-one is and that there are many grey areas in life that are subjective. In all, you might not actually be narcissistic...