Sometimes, having too much empathy means that you tolerate stupidity, low performance, laziness, and hold yourself back from your potential.
We all take certain cues from our surroundings, and those who surround us. And it's inevitable that if you're in a mediocre environment, some of that rubs off on you over time. E.g. people who are dragging you down in terms of performance, ideas, not being "on it" and not at the top of their game, if they ever were. You start forgetting that you're better than this.
Given enough explanation, anything becomes reasonable. Being too empathetic to people sometimes means that you have to put up with excuses of all the bullshit that goes on in average people's lives, and how poorly people manage their own thoughts, affairs, and actions. Your coworker is dealing with relationship issues, can't deal with stress, unable to think clearly, blah blah blah.
Most of us want, at some level, to see what's on the next level, get a taste of being a rock star, a CEO, something special? Not just to be an empathetic person who understands why everyone has issues and has to put up with it?
Sometimes, not having empathy is necessary to make others perform, do what they don't want to do, and achieve beyond the every day. Work is about doing things you don't want to do. If everything is empathetic, why even bother doing things you don't want to do?
Now, the clever amongst us will learn how to use empathy as a tool to get people to feel you relate to them, and motivate them to do their shit. Or otherwise, at least recognize that you're not surrounded by geniuses, and change your surroundings to where you're challenged and maybe the one underperforming. Of course, your ability to use these factors/skills depends on whether you're stuck with a group of colleagues or have freedom to move around (i.e. how much you can burn bridges/create new ones or are stuck with people).
But overall, use some amount of caution with empathy. It's liberal and polite to be empathetic. But too much empathy and you can sometimes be dragged into a pit of mediocrity yourself.